Christian Herdtweck [Mon, 15 Apr 2019 08:09:20 +0000]
Change version to 1.5
Christian Herdtweck [Mon, 15 Apr 2019 08:09:51 +0000]
Remove outdated check script. Will bring back as adapted copy from QA code
Christian Herdtweck [Tue, 9 Apr 2019 12:20:59 +0000]
Increase version to 1.5 (alpha)
Christian Herdtweck [Tue, 9 Apr 2019 09:53:34 +0000]
Merge branch 'py3.7-ssl-certs'
Christian Herdtweck [Tue, 9 Apr 2019 08:33:23 +0000]
Take into account that certs are not issued to "localhost"
Use FQDN instead
Christian Herdtweck [Tue, 9 Apr 2019 08:20:55 +0000]
Make certificate checks optional, enable per default
Also give an option to give an extra certificate to accept as .pem
Christian Herdtweck [Tue, 9 Apr 2019 08:19:16 +0000]
Fix error type returned from failed POST
Plamen Dimitrov [Tue, 26 Mar 2019 03:53:52 +0000]
Allow HTTPS connections with self-signed certificates
We use such certificates on the web surface of the Intra2net system.
Christian Herdtweck [Tue, 9 Apr 2019 09:52:43 +0000]
Improve output of unittest
Christian Herdtweck [Tue, 9 Apr 2019 09:50:53 +0000]
Create dist for python3.7 per default (and add help)
Both Intra2net systems and avocado are now running on python3.7
Christian Herdtweck [Tue, 9 Apr 2019 09:50:11 +0000]
Fix bad end of iterator and unused variable
The StopIteration is a real error in python 3.7
Christian Herdtweck [Tue, 9 Apr 2019 09:46:04 +0000]
Merge branch 'py3.7-subprocess'
Christian Herdtweck [Tue, 9 Apr 2019 09:43:24 +0000]
Remove unittests for removed function
Plamen Dimitrov [Tue, 26 Mar 2019 07:40:07 +0000]
Remove obsoleted comment in the arnied wrapper docstring
Plamen Dimitrov [Tue, 26 Mar 2019 07:36:13 +0000]
Drop emulation of py3.5+ subprocess module for python 3.3
The Intra2net system now relies on python 3.7 ridding us of all
this extra complexity.
Plamen Dimitrov [Tue, 26 Mar 2019 07:30:50 +0000]
Define new capture output flag for python 3.7 which is false by default
This is a bad indicator of the backwards compatibility of minor versions
since adding the flag with a default like this would probably break a
lot of code down there. Sometimes it seems that python developers are
not really concerned with how many things they break in their minors.
Plamen Dimitrov [Mon, 18 Feb 2019 12:28:56 +0000]
Remove the global TODO with outdated information
We can use Mantis BT, Trello, or other system for better
tracking of tasks.
Christian Herdtweck [Tue, 12 Feb 2019 10:30:37 +0000]
Make unittest output less distracting
That blinking really catches my eye every time, quite distracting.
Also limit output to a single line.
Christian Herdtweck [Tue, 12 Feb 2019 10:21:46 +0000]
Improve output of wait_for_email_transfer
Most of all, warn if timeout was reached without finishing email transfer
Christian Herdtweck [Tue, 12 Feb 2019 09:20:54 +0000]
Run unittests in make_dist.py; update doc
Christian Herdtweck [Tue, 12 Feb 2019 08:54:57 +0000]
Add release argument to make_dist.py
This should simplify introducing a proper labeling of development versions
of pyi2ncommon (the -avocado-test-X naming scheme did not prove successful)
Christian Herdtweck [Mon, 11 Feb 2019 09:27:58 +0000]
Re-run with LANG=C in unittest runner
Christian Herdtweck [Mon, 11 Feb 2019 12:20:04 +0000]
Merge branch 'cnfvar-autodetect-bytes'
Christian Herdtweck [Mon, 11 Feb 2019 10:55:27 +0000]
Create unittest for read_cnf_json(non-ascii-str)
Christian Herdtweck [Mon, 11 Feb 2019 10:54:50 +0000]
Fix unittest if run from non-unicode shell
When opening files (like os.devnull) in text mode, python uses the shell's
default encoding. In case LANG=C or when redirecting (for example), that
is just 'ascii', so writing non-ascii strings to /dev/null fails with
UnicodeError
Christian Herdtweck [Mon, 11 Feb 2019 10:51:54 +0000]
Add and fix and few CNF_VAR-related doc strings
Christian Herdtweck [Mon, 11 Feb 2019 09:50:13 +0000]
Auto-detect str/bytes when dumping cnf vars
This is the second attempt to solve the problem when to "print" byte strings
and when to use regular unicode strings.
One problem is that sys.stdout can be a text stream or a byte stream
depending on shell environment.
Christian Herdtweck [Thu, 7 Feb 2019 15:45:54 +0000]
Merge branch 'make-dist-improvements'
Christian Herdtweck [Mon, 28 Jan 2019 10:26:18 +0000]
Avoid modifying wrong rpm from make_dist
Bughunting before was complicated by usage of wrong rpm files. Make tests
more strict to avoid finding wrong rpm
Christian Herdtweck [Mon, 28 Jan 2019 09:56:55 +0000]
Require zipfile35 only for rpms for py<3.5
Christian Herdtweck [Mon, 28 Jan 2019 08:45:49 +0000]
Document build machine requirement in README
Christian Herdtweck [Mon, 28 Jan 2019 08:26:13 +0000]
Allow specify py version for make_dist
make_dist.py had assumed that py3.3 and py3.6 are installed on local machine.
Document that assumption and allow to deviate from it
Plamen Dimitrov [Fri, 25 Jan 2019 11:37:49 +0000]
add a minimal setup guide in order to build the RPMs
Christian Herdtweck [Thu, 7 Feb 2019 15:43:19 +0000]
Merge branch 'cnfvar-byte-fixes'
Plamen Dimitrov [Mon, 21 Jan 2019 12:34:43 +0000]
Be more precise about the binary in sysmisc cheat reboot docstring
Plamen Dimitrov [Fri, 25 Jan 2019 17:32:03 +0000]
Pass byte stream to the cnfvar from the dynamic cnf setting
Plamen Dimitrov [Fri, 25 Jan 2019 11:18:33 +0000]
Ignore some PyDev configuration in the git tree
Plamen Dimitrov [Fri, 25 Jan 2019 17:33:15 +0000]
Deprecate irrelevant autotest comments in the cnfvar
Christian Herdtweck [Thu, 7 Feb 2019 15:35:33 +0000]
Merge branch 'mail_utils-for-avocado-emailfilter'
Plamen Dimitrov [Fri, 25 Jan 2019 17:31:40 +0000]
Fix cnfvar case of writing bytes from the io module
Juliana Rodrigueiro [Wed, 6 Feb 2019 10:57:52 +0000]
Use a more robust condition in wait_for_email_transfer
This solution uses 'postqueue -j' command, which will produce an
empty list for stdout only when there are no more emails in the
queue. We become independent of the format of the output (byte,
string, specific characters, etc)
Christian Herdtweck [Thu, 24 Jan 2019 15:50:39 +0000]
Create unittest for new mail_utils feature
Christian Herdtweck [Thu, 7 Feb 2019 15:22:26 +0000]
Merge branch 'arnied-fix-remote-scheduling'
Christian Herdtweck [Thu, 24 Jan 2019 15:43:08 +0000]
Create new mail_utils option: get headers+attachments
Often I parse and transfer big email message when actually
I only want the headers plus the attachment file names.
Create an option for this in mail_utils.parse_mail_file
Christian Herdtweck [Thu, 7 Feb 2019 15:20:36 +0000]
Merge branch 'call-helper-no-default-split'
Samir Aguiar [Fri, 14 Dec 2018 18:28:00 +0000]
Fix remote scheduling of programs
When scheduling a program on a remote VM, the schedule file was
not being sent to the VM. Also, when removing previously schedule
files rm was waiting for confirmation since we didn't pass it
the `-f` option.
Christian Herdtweck [Tue, 4 Dec 2018 12:36:44 +0000]
pylint/pydocstyle compliance for mail_utils
- Wrapped long lines
- Added/Modified doc strings
- Replace single-letter variable names
Christian Herdtweck [Thu, 7 Feb 2019 15:15:45 +0000]
Merge branch 'cnfvar-fixes'
Christian Herdtweck [Fri, 21 Dec 2018 15:25:17 +0000]
test_call_helpers: create unittests for subprocess_run
Check that subprocess_run is compatible with python3.5 subprocess.run
Samir Aguiar [Fri, 14 Dec 2018 18:27:06 +0000]
Overwrite run_cmd timeout in wait_for_run
The default timeout of 60 seconds might not be enough
Christian Herdtweck [Tue, 2 Oct 2018 11:02:07 +0000]
Create new mail util function to get body text(s)
Philipp Gesang [Thu, 7 Feb 2019 14:42:21 +0000]
tidy up cnfvar.py unit tests
This batch of unit tests was disabled since the switch to Python3
on account of minor encoding troubles. Reactivate it and ensure
encoding and escaping are handled properly.
Christian Herdtweck [Fri, 21 Dec 2018 15:24:33 +0000]
test_call_helpers: fix Comments
Samir Aguiar [Fri, 14 Dec 2018 18:24:16 +0000]
Add timeout support to run_cmd
When calling shell commands on a VM we have a default timeout
of 60 seconds (in the session functions designed to run remote
commands). However, some arnied_wrapper functions such as
`wait_for_run` (which calls `arnied_helper
--wait-for-program-timeout`) require a larger timeout, so we
need to be able to overwrite the default.
Christian Herdtweck [Fri, 15 Dec 2017 15:17:27 +0000]
Add function to decode non-ascii attachment names from mails
Christian Herdtweck [Fri, 21 Dec 2018 15:22:32 +0000]
call_helpers: Make default behaviour of subprocess_run compatible
subprocess_run just calls call_and_capture which has different default
settings (return text instead of bytes, split lines).
However, subprocess_run should be a drop-in replacement of the python3.5
subprocess.run function, so should behave the same way.
This little trick fixed e.g. calls to "get_cnf -j" called from cnfvar.py
on python 3.3 (current intra2net system version)
Philipp Gesang [Thu, 7 Feb 2019 14:25:04 +0000]
accept both bytes and str for cnfvar readers
Make both the CNF format and the JSON readers handle bytes and
str transparently.
(Note there is still a deplorable asymmetry in the routines such
that the JSON path will convert to *str* whereas the CNF part,
assuming single-byte encoded data, will convert to *bytes*. This
is necessary for compatibility reasons until the Intranator gets
a more recent Python interpreter.)
Philipp Gesang [Thu, 7 Feb 2019 14:15:10 +0000]
allow selecting individual tests in unittest wrapper
Extend run_unittests.sh so it doesn't necessarily always run
everything:
$ ./run_unittests.sh # -> as before
$ ./run_unittests.sh test.test_cnfvar # run test/test_cnfvar.py
$ ./run_unittests.sh test_cnfvar # run test/test_cnfvar.py
Philipp Gesang [Thu, 7 Feb 2019 13:45:40 +0000]
handle data escaping properly in cnf vars
libcnffile has an additional step that strips escaping from
var data so this:
42 MY_VAR,0: "foo\"bar\"baz"
is actually a valid CNF_VAR with a data member equals ``foo"bar"baz''.
Till now, cnfvar.py would reject such a line because of the
stricter matching. Since escaped strings are ubiquitous in JSON
encoded data, this must be handled properly. (Ironically this is
not an issue with the JSON interface due to the builtin string
escaping rules of the format.)
Imitate the behavior of libcnffile so properly escaped double
quotes don't terminate the string matching. Lucky for us we can
get away with extending the line regex with an alternating
pattern in a non-matching group ``(?:...''.
Christian Herdtweck [Thu, 7 Feb 2019 11:59:17 +0000]
Create unittests for case changes of cnfvar varnames
Philipp Gesang [Thu, 7 Feb 2019 13:16:44 +0000]
normalize cnfvars for JSON output too
Reported by Christian: The normalization pass that ensures
varnames are uppercase was missing from the JSON based routines
which caused them to be rejected by set_cnf(). Implement it by
passing a normalized copy of the CNF tree to the json writer.
Christian Herdtweck [Thu, 31 Jan 2019 15:06:49 +0000]
Fix debug output line in dial.py
Christian Herdtweck [Thu, 31 Jan 2019 13:10:20 +0000]
Convert cnfvar names to lowercase on json import
read_cnf already converts all variable names to lower case. Do the same for
read_cnf_json to get consistent behaviour. SimpleCnf already relies on this.
Christian Herdtweck [Thu, 31 Jan 2019 13:08:58 +0000]
Remove unnecessary creation of reference
Christian Herdtweck [Thu, 31 Jan 2019 13:08:22 +0000]
Correct doc in simple_cnf
Christian Herdtweck [Tue, 22 Jan 2019 09:59:45 +0000]
Merge branch 'less-unittest-clutter'
Christian Herdtweck [Wed, 14 Nov 2018 10:34:41 +0000]
Disable debug output in log_read
Fixed based on Plamen's review.
Christian Herdtweck [Wed, 5 Dec 2018 15:27:18 +0000]
Create 2 pyi2ncommon rpms: for py3.3 and py3.6
The python version, for which pyi2ncommon is installed on target machine,
is fixed in the rpm. It defaults to same version as build host. Managed
to change that by creating setup.cfg file dynamically while running
setup.py . With help of that script, can now create one version for
autotest (py3.6) and one for I2n systems (py3.3)
Christian Herdtweck [Wed, 5 Dec 2018 15:22:41 +0000]
Do not require fixed python3 version for pyi2ncommon
Christian Herdtweck [Wed, 14 Nov 2018 10:33:26 +0000]
Reduce output from simple_cnf test to warnings
Christian Herdtweck [Wed, 14 Nov 2018 09:53:10 +0000]
Replace deprecated logging function with equivalent
Christian Herdtweck [Wed, 14 Nov 2018 09:53:02 +0000]
Replace deprecated unittest function with equivalent
Christian Herdtweck [Wed, 14 Nov 2018 08:53:08 +0000]
Ensure all opened files/sockets are closed in mail_utils
Philipp Gesang [Thu, 25 Oct 2018 08:47:42 +0000]
bump minor version to keep track of changes
Christian Herdtweck [Mon, 5 Nov 2018 11:22:07 +0000]
Merge branch 'util-resurrection'
Christian Herdtweck [Mon, 5 Nov 2018 10:30:40 +0000]
Fix doc strings for text_helpers (except color stuff)
Thanks to Plamen for hint in review
Christian Herdtweck [Mon, 5 Nov 2018 10:23:34 +0000]
Fix doc string for sysmisc.cd()
Thanks to Plamen for hint
Christian Herdtweck [Mon, 29 Oct 2018 08:56:35 +0000]
Create unittest for size_str
Christian Herdtweck [Mon, 29 Oct 2018 08:56:27 +0000]
Bring back size_str text helper
Had been removed since it was not needed any more. Is needed again, so may
be relevant after all
Christian Herdtweck [Mon, 29 Oct 2018 08:41:47 +0000]
Create unit tests for cd context manager
Christian Herdtweck [Mon, 29 Oct 2018 08:41:37 +0000]
Bring back cd context manager
This has been in pyi2ncommon before but was removed since it was only used
in one place (which exists no more). Now needed it again, so bring it back.
Christian Herdtweck [Mon, 5 Nov 2018 11:21:17 +0000]
Merge branch 'cnfvar-dev'
Philipp Gesang [Mon, 5 Nov 2018 09:47:49 +0000]
fix misleading error message in simple_cnf
Philipp Gesang [Mon, 5 Nov 2018 09:39:12 +0000]
normalize varname as cnfvar.py expects it
When cnfvar reads variables, it normalizes the names to
lowercase. Only when emitting classic CNF_FILE style output
the names are uppercased for set_cnf.
Philipp Gesang [Mon, 5 Nov 2018 09:12:35 +0000]
support comparison of SimpleCnf() with non-SimpleCnf() objects
Since __eq__ operations are polymorphic, crashing on different
``types'' is not a good idea.
Philipp Gesang [Mon, 5 Nov 2018 08:58:43 +0000]
validate SimpleCnf inputs
The cnfvar functions cannot handle ``tuple'' inputs since the
inputs are variable length. Make simple_cnf pass a list instead
and reject non-list inputs in the ctor.
Philipp Gesang [Mon, 5 Nov 2018 08:21:49 +0000]
restore simple_cnf/add_single API
Restore an API that vanished at some point despite there still
being users.
This is commit
fa747d25a7c47a1ceef165c4e1667b9755c9da0d on
autotest/i2n. The original commit message in full:
In
commit
5f6735c41fc769ac9ba6ee62754ab2b7da3d68d2
Author: Christian Herdtweck <christian.herdtweck@intra2net.com>
Date: Wed Apr 5 18:14:02 2017 +0200
Extend SimpleCnf.add_single to SimpleCnf.add
this ApI was folded into .add() without providing an overload for
downstream consumers. Provide the symbol again as a thin wrapper
around .add() and hope for the behavior to remain unaffected.
Philipp Gesang [Tue, 24 Jul 2018 13:46:54 +0000]
fix cnfvar crash on end inside child vars
The recursive call to parse_cnf_children() may also return None
if it hit EOF.
Philipp Gesang [Fri, 8 Dec 2017 12:34:46 +0000]
lowercase all incoming varnames
The CNF store is case-indifferent regarding variable names when
reading from it, so just normalize all of them when internalizing
variables.
Philipp Gesang [Tue, 5 Dec 2017 12:47:45 +0000]
when emitting CNF lines, force capitalization of variable names
Philipp Gesang [Tue, 5 Dec 2017 12:46:47 +0000]
add cnfvar helper for counting CNF trees
Philipp Gesang [Tue, 5 Dec 2017 10:55:24 +0000]
allow parent-less renumbering of CNF_VARs
When more control is needed over the line numbers a CNF_VAR and
its children are assigned, the *parent* linenumber can be used to
pass an offset. Add a parameter *toplevel* to prevent the parent
index from being set.
Philipp Gesang [Tue, 5 Dec 2017 10:38:40 +0000]
loosen criteria on CNF object roots
Single CNF_VARs are technically malformed since CNF file is
always a list of vars. However, any isolated CNF line can be
trivially converted to a valid CNF_VAR by wrapping it in a list
ctor, so do that automatically when determining the root of a
CNF_VAR.
Philipp Gesang [Tue, 5 Dec 2017 10:02:18 +0000]
fix renumbering in cnfvar.py
Philipp Gesang [Fri, 1 Dec 2017 15:14:56 +0000]
avoid formatting binary strings
Postpone bytes conversion of CNF-encoded objects until the last
moment to work around missing interpolation of bytes objects in
Python 3.3 (the version still in use on the Intranator).
Reference: https://bufs.python.org/issue3982
Christian Herdtweck [Wed, 24 Oct 2018 09:51:37 +0000]
Update version to 1.3
Plamen Dimitrov [Tue, 30 Oct 2018 15:12:27 +0000]
Use HTTP firewall ruleset option starting from I2N release 6.5.0
Christian Herdtweck [Wed, 24 Oct 2018 15:30:27 +0000]
Fix encoding issue when building rpm
This issue never occured in regular shells because usually LOCALE and
system encoding and similar is set.
Christian Herdtweck [Wed, 24 Oct 2018 14:40:22 +0000]
Add CONTRIBUTING guide with rules for contributors
Christian Herdtweck [Wed, 24 Oct 2018 13:18:52 +0000]
Make debug print in test optional