# on this file might be covered by the GNU General Public License.
"""
-follow.py: follow process output, log files and pipes using select and poll
+follow process output, log files and pipes using select and poll
Main class is Follower which does the polling and selecting, it is best used
- in a with-statement as follows:
+in a with-statement as follows::
-with follow('/var/log/messages') as flwr
- for line in flwr:
- do_something_with(line)
+ with follow('/var/log/messages') as flwr
+ for line in flwr:
+ do_something_with(line)
This will read the given file and yield its contents line-by-line until the end
of the file. It will then wait for new additions to the file and provide the
new lines newly instantaneously
Things to note:
+
* all data must be line-based!
-* will only work on Linux (except for sockets maybe)
-* create in py2 but try to stay py3-compatibel
+* will only work on Linux (except for sockets maybe).
+* create in py2 but try to stay py3-compatible.
[START: not implemented yet]
+
If following a log file, a LogParser can be attached that auto-detects some of
- the log's structure (like date and time fields, log levels and sources) from
- its first few lines.. This can be used anlogously. Of course, you can also
- specify the log's structure (to be implemented before the auto-detect...).
+the log's structure (like date and time fields, log levels and sources) from
+its first few lines.. This can be used anlogously. Of course, you can also
+specify the log's structure (to be implemented before the auto-detect...)::
+
+ with follow_log('/var/log/messages') as log_flwr:
+ for content in log_flwr:
+ do_something_with(content.datetime, content.log_level, content.text)
-with follow_log('/var/log/messages') as log_flwr:
- for content in log_flwr:
- do_something_with(content.datetime, content.log_level, content.text)
[END: not implemented yet]
A Follower is an iterator, which means you can do lots of cool things with it,
- including (see also itertools package, itertool recipies, "Functional
- Programming Howto"):
+including (see also itertools package, itertool recipies, "Functional
+Programming Howto")::
+
+ # ignore source and description:
+ for _, _, text_line in my_follower:
+ do_something_with(line)
-# ignore source and description:
-for _, _, text_line in my_follower:
- do_something_with(line)
+ # enumerate:
+ for line_number, line in enumerate(my_follower)
+ do_something_with(line, line_number)
-# enumerate:
-for line_number, line in enumerate(my_follower)
- do_something_with(line, line_number)
+ # combine with other iterator:
+ for line, other_data in zip(my_follwer, other_iter)
+ do_something_with(line, other_data)
-# combine with other iterator:
-for line, other_data in zip(my_follwer, other_iter)
- do_something_with(line, other_data)
+ # filter:
+ for line in my_follower if test(my_func):
+ do_something_with(line)
-# filter:
-for line in my_follower if test(my_func):
- do_something_with(line)
+ # tee:
+ iter1, iter2 = itertools.tee(my_follower)
+ --> can read from both (but each line is given to only one of them)
-# tee:
-iter1, iter2 = itertools.tee(my_follower)
---> can read from both (but each line is given to only one of them)
+ # for each element, peek at the next one to help do the right thing
+ for line, next_line in pairwise(my_follower):
+ do_something_with(line, peek_next_line=next_line)
-# for each element, peek at the next one to help do the right thing
-for line, next_line in pairwise(my_follower):
- do_something_with(line, peek_next_line=next_line)
+ # create new iterator or generator
+ for line in my_follwer:
+ some_result = do_something_with(line)
+ yield some_result
-# create new iterator or generator
-for line in my_follwer:
- some_result = do_something_with(line)
- yield some_result
+NOT possible::
-NOT possible: len(my_follower), Follower(my_file, my_file) # (twice the same)
+ len(my_follower)
+ Follower(my_file, my_file) # (twice the same)
-Christian Herdtweck, Intra2net, July 2015
-(c) Intra2net AG 2015
+.. codeauthor:: Christian Herdtweck, christian.herdtweck@intra2net.com
"""
from __future__ import print_function
Will guess if args are just sources or also descriptions of these.
All of these are possible:
- Follower(src)
- Follower(src, desc)
- Follower(src1, src2, src3)
- Follower(src1, desc1, src2, desc2, src3, desc3)
- Follower(src1, src2, desc2, src3)
- Follower(src, desc_ignored, desc_used) # warn but accept
- Follower(single_list_of_sources_and_descs)
+
+ * Follower(src).
+ * Follower(src, desc).
+ * Follower(src1, src2, src3).
+ * Follower(src1, desc1, src2, desc2, src3, desc3).
+ * Follower(src1, src2, desc2, src3).
+ * Follower(src, desc_ignored, desc_used) # warn but accept.
+ * Follower(single_list_of_sources_and_descs).
Not possible:
- Follower()
- Follower(desc, src)
+
+ * Follower().
+ * Follower(desc, src).
Descriptions must be strings, they identify the sources in generated
- output and are used in error messages
+ output and are used in error messages
+
Sources must be file handles, open pipes or open sockets (or anything
- else that gives a reasonable fileno(), so help of the select module)
+ else that gives a reasonable fileno(), so help of the select module)
+
Sources are not closed!
other_args can include flags and timeout
git://developer.intra2net.com / pyi2ncommon / commitdiff