With iterpipes, python is ready to replace bash for scripting. Really.
This has been a pet peeve of mine for years: programming shell scripts suck. They are ugly and error prone. The only reason why we still do it? There is no real replacement.
Or at least that was the case, until today I met iterpipes at python.reddit.com
Iterpipes is "A library for running shell pipelines using shell-like syntax" and guess what? It's brilliant.
Here's an example from its PYPI page:
# Total lines in *.py files under /path/to/dir,
# use safe shell parameters formatting:
>>> total = cmd(
... 'find {} -name {} -print0 | xargs -0 wc -l | tail -1 | awk {}',
... '/path/to/dir', '\*.py', '{print $1}')
>>> run(total | strip() | join | int)
315
Here's how that would look in shell:
find /path/to/dir -name '*.py' -print0 | xargs -0 wc -l | tail -1 | awk '{print $1}'
You may say the shell version looks better. That's an illusion caused by the evil that is shell scripting: the shell version is buggy.
Why is it buggy? Because if I control what's inside /path/to/dir
I can make that neat little shell command fail [1], but at least in python I can handle errors!
Also, in most versions you could attempt to write, this command would be unsafe because quoting and escaping in shell is insane!
The iterpipes version uses the equivalent of SQL prepared statements which are much safer.
It's nearly impossible to do such a command in pure shell and be sure it's safe.
Also, the shell version produces a string instead of an integer, which sucks if you intend to do anything with it.
And the most important benefit is, of course, not when you try to make python act like a shell, but when you can stop pretending shell is a real programming language.
Consider this gem from Arch Linux's /etc/rc.shutdown
script. Here, DAEMONS is a list of things that started on boot, and this script is trying to shut them down in reverse order, unless the daemon name starts with "!":
# Shutdown daemons in reverse order let i=${#DAEMONS[@]}-1 while [ $i -ge 0 ]; do if [ "${DAEMONS[$i]:0:1}" != '!' ]; then ck_daemon ${DAEMONS[$i]#@} || stop_daemon ${DAEMONS[$i]#@} fi let i=i-1 done
Nice uh?
Now, how would that look in python (I may have inverted the meaning of ck_daemon)?
# Shutdown daemons in reverse order
for daemon in reversed(DAEMONS):
if daemon[0]=='!':
continue
if ck_daemon(daemon):
stop_daemon(daemon)
Where stop_daemon used to be this:
stop_daemon() { /etc/rc.d/$1 stop }
And will now be this:
def stop_daemon(daemon):
run(cmd('/etc/rc.d/{} stop',daemon))
So, come on, people, we are in the 21st century, and shell scripting sucked in the 20th already.
definitivamente te la pasas revisando pipy.....
No, estoy subscripto a python.reddit.com ... mucho menos laburo! ;-)
Excellent post!
Bash is obsolete. Long live Python!
Awesome thank you!
Just today I was ranting about bash and how it sucks so much!
Thank you for sharing.
IPython is also great for on-the-fly system administration:
>>> files = !ls *.py
etc.
Thanks for your review :) The library is still in active development and definitely requires more documentation, testing, cleaning up its API, etc.
Your example with Arch Linux init scripts is quite interesting, because scripting in Arch is one of my use cases for the library.
BTW, in run(cmd('/etc/rc.d/{} stop',daemon)), you should use call or check_call instead of run as run returns Iterable -> Iterable and here you just need to force immediate execution of a command. As I've already said, more docs are needed.
Thanks for the correction. That's what happens when I don't test the code I post!
I'm a Python lover and this post is a WOW! experience. Python scripts look **so much** better than Bash scripts do!
Also the *error handling* is a plus point of why to replace a dynamic programming language, like Python, with an old-school scripting language like Bash.
Maybe this was a tedious task when using the `sys` and `os` or `subprocess` module for executing commands and let's not talk about pipelining... that was very ugly. But with `iterpipes` this is a complete new experience.
Thanks for sharing!
There is a nice, long discussion of this post here (in czech): http://is.gd/6fstv
Google translation: http://is.gd/6a1bC
Hi! You could be interested on Scripy. It has been built to replace to bash scripts, it's integrated with logging (all shell commands can be logged), to lets edit files (more easy than in bash), and more...
http://pypi.python.org/pypi...
http://bitbucket.org/ares/s...
Very interesting!
In your first example you use pipes in an infix notation; I can't find that iterpipes supports this, was it dropped?
this is really interesting viewpoint on the subject i might add
Man ... Beautiful . Amazing ... I will bookmark your website and use the your RSS feed also
Your blog has the same post as another author but i like your better