DPDK is a fantastic piece of software. I’ve used it both for
work and in my hobby projects (yes, I crunch packets as a hobby, say
what you will about me!) and it works great. My only grievance with it
has always been the complicated build system it forces on you. It’s
ugly and inflexible and it sometimes drives you crazy.
Yesterday, I saw that DPDK is in Ubuntu’s repositories. It took me
some time to realize wha that means. I was looking for the value of
the RTE_SDK variable when it clicked in my head; I could just use the
damn thing like any normal library; with -l, -L, -I and all
that. I just needed to add a -msse4.2 flag for the compilation to
work properly (I’ve been adding that flag to the TOOLCHAIN_CFLAGS
make variable on some of my machines to make DPDK compile anyway).
Does it have any drawbacks? I have no idea, as I never managed to
penetrate the many layers of DPDK make files to see what they really
do. It’s quite possible that this is not as efficient as compiling
DPDK yourself so it can detect and use the full capabilities of your
machine, but at the very least I learned a thing or two about building
DPDK by looking at the the Git repository of the Ubuntu
maintainers. Just do a git clone
https://git.launchpad.net/~ubuntu-server/dpdk. Switch to the
ubuntu-xenial branch to see the interesting bits.
This seems to be a Canonical endeavor, so thank you kind folks in
I’m a huge fan of rtorrent. It’s simple, lean and elegant. I have
so far only used it as an interactive BitTorrent client inside
screen/tmux or as a batch downloader with a watch directory.
There is more to rtorrent however. It supports XML-RPC which means you
can control it programmatically. As I was in dire need of a client I
can manipulate from a script, I spent some time today to setup
rtorrent correctly and call it remotely.
First we’re going to need something like this line in our
scgi_port = localhost:5005
You can also make rtorrent listen to a UNIX socket:
scgi_local = /tmp/rtorrent-listening-here.sock
After that, we need a web server to forward our requests to rtorrent,
and responses back to us. The web server will be communicating with
rtorrent through SCGI. I use nginx for this purpose. Here’s
the relevant nginx config for this:
After this, reload nginx config, make sure rtorrent is running, and
you’re ready to go.
In order to test your setup, or contact rtorrent from a shell script,
you can use the xmlrpc utility accompanied by the libxmlrpc
library. You can get this in Ubuntu by running sudo apt-get install
libxmlrpc-core-c3-dev. After that, run this:
xmlrpc localhost:8000 system.listMethods
If everything is done correctly, you will get a list of RPC methods
supported by rtorrent.
There are a lot more commands. Try some of them for yourself. I admit
that it’s difficult to find out how each command works this way, but
in the absence of a comprehensive documentation, it seems to be the
only way. There are a few examples here.
Complete Vagrant Setup
I have put the complete setup for use with vagrant in this
github repository. Simply clone the repo, and run vagrant up in
its directory. After the VM is up, you can access rtorrent on the
forwarded port 8080:
I just uploaded a Python/asyncio-based UDP BitTorrent tracker to my
github account. It’s called pybtracker and you can find it
You can install pybtracker using pip by running pip3 install
pybtracker. You’ll need Python 3.5. After installing pybtracker you
can simply run it like pybtracker -b 127.0.0.1:8000 -O. An
interactive client is also included which can be used by running
pybtracker-client udp://127.0.0.1:8000 (update server address as you
Say you are writing a test case for an AsyncIO-based network
function. You want to write a test server and have the code being
tested connect to it. You can choose a port number and hope it’s not
taken when the test is run, or you can have a free port chosen for you
each time the test is run. Simply pass 0 as the local port number:
Afterwards, you can query the transport object for the port number
chosen for you:
And that’s it; now you know the port number. I’m not sure if using a
member variable with an underscore at the start of its name is the
best or the official way to do this but it’s the best method I’ve
found so far.
April 2016 Update: Native support for Python 3.5 has finally landed
in Emacs development branch. You don’t need this patch anymore.
Native co-routine support in Python 3.5 is really cool. So cool in
fact that it finally made me switch from Python 2. I never was a big
fan of Python 3, but that’s another story. The point is I’m finally
ready to write some Python 3 code and I find out that my favorite
editor (Emacs) does not support the new functionality. I don’t need
anything fancy really. I just need proper syntax coloring, proper
indentation and working navigation commands.
The indentation seems to be working fine in Emacs 24 already, but the
other two are not. The best I could find was the patch accompanying
this message on the Emacs mailing list. All I needed to do was to
get Emacs source code (git clone
git://git.savannah.gnu.org/emacs.git; there, I probably just spared
you one Google search!), make some minor modifications (see below) and
apply the patch (git apply py35.diff).
One small thing missing in the patch was proper coloring for the new
keyword await. Another problem was that the file locations are not
the same as in the git head. I’ve made the proper adjustments to make
it possible to apply the patch to the current master branch. You can
download the updated patch from the link below.
I should really thank Lele Gaifax from the Emacs mailing list. You
made my day stranger!