Pull request to allow supervisor to send arbitrary signals to processes

https://github.com/Supervisor/supervisor/pull/477

Let’s you do stuff like:

$ supervisorctl
cat:0                            RUNNING   pid 57305, uptime 0:00:07
cat:1                            RUNNING   pid 57304, uptime 0:00:07
cat:2                            RUNNING   pid 57307, uptime 0:00:07
cat:3                            RUNNING   pid 57306, uptime 0:00:07
cat:4                            RUNNING   pid 57308, uptime 0:00:07
dog:0                            RUNNING   pid 57300, uptime 0:00:07
dog:1                            RUNNING   pid 57299, uptime 0:00:07
dog:2                            RUNNING   pid 57302, uptime 0:00:07
dog:3                            RUNNING   pid 57301, uptime 0:00:07
dog:4                            RUNNING   pid 57303, uptime 0:00:07
supervisor> help signal
signal <signal name> <name>	      Signal a process
signal <signal name> <gname>:*        Signal all processes in a group
signal <signal name> <name> <name>    Signal multiple processes or groups
supervisor> signal 1 dog:3 dog:4
dog:3: signalled
dog:4: signalled
supervisor> signal HUP dog:3 dog:4
dog:3: signalled
dog:4: signalled
supervisor> signal HUP dog:*
dog:1: signalled
dog:0: signalled
dog:3: signalled
dog:2: signalled
dog:4: signalled
supervisor> signal USR1 dog:1 dog:2
dog:1: signalled
dog:2: signalled

Also, if you can’t wait for supervisor to support this, the mr.laforge package supplies a supervisor plugin that can be used to send signals to processes:

$ supervisorctl kill HUP nginx

That said, it would be nice to have this built into supervisor…

Easier way to build _bsddb for OS X

I blogged previously about how to build _bsddb for Python 2.6 by hacking the Python source code to accommodate changes in berkeley-db.

Here’s an easier way that doesn’t require hacking source code.

Install berkeley-db using Homebrew:

brew install berkeley-db

Install the bsddb3 module, pointing it at an installation of berkeley-db:

sudo \
    BERKELEYDB_DIR=/usr/local/Cellar/berkeley-db/5.3.28 \
    pip-2.6 install bsddb3

Simply copy it into the appropriate place as _bsddb.so. This is super hacky and will fail if the interfaces of these two ever change, but for now for my limited purpose, it seems to work:

cp \
    /Library/Python/2.6/site-packages/bsddb3/_pybsddb.so \
    /Library/Frameworks/Python.framework/Versions/2.6/lib \
      /python2.6/lib-dynload/_bsddb.so

Enjoy:

$ python2.6
Python 2.6.7 (r267:88850, Oct 11 2012, 20:15:00)
[GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import bsddb
>>> dir(bsddb)
['MutableMapping', '_DBWithCursor', '_DeadlockWrap', '__builtins__', '__doc__', '__file__', '__name__', '__package__', '__path__', '__version__', '_bsddb', '_checkflag', '_db', '_iter_mixin', '_openDBEnv', 'absolute_import', 'btopen', 'collections', 'db', 'dbutils', 'error', 'hashopen', 'os', 'ref', 'rnopen', 'sys']

Building _bsddb for Python 2.6 on OS X 10.8.5

It’s well-known that the Python that comes with OS X machines typically doesn’t have _bsddb.

I downloaded Python 2.6.8 and attempted to build it and it wouldn’t build the _bsddb module.

Here’s what I did to fix this.

I used Homebrew to install berkeley-db4. For me, this grabbed version 4.8.30.

brew install berkeley-db4

Edited Modules/_bsddb.c and replaced all instances of DB_XIDDATASIZE with DB_GID_SIZE. I removed lines that had DB_XA_CREATE.

Compiled and linked with:

gcc -fno-strict-aliasing -fno-common -dynamic -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/Cellar/berkeley-db4/4.8.30/include/ -I/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c /Users/marca/src/Python-2.6.8/Modules/_bsddb.c -o _bsddb.o
gcc -bundle -undefined dynamic_lookup _bsddb.o -L/Library/Frameworks/Python.framework/Versions/2.6/lib -L/usr/local/lib -ldb -o _bsddb.so -Wl,-search_paths_first

This created the file Modules/_bsddb.so.

At this point, I can import if I run python in the Modules directory where I built _bsddb.so:

$ python2.6
Python 2.6.7 (r267:88850, Oct 11 2012, 20:15:00)
[GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import bsddb
>>> dir(bsddb)
['MutableMapping', '_DBWithCursor', '_DeadlockWrap', '__builtins__', '__doc__', '__file__', '__name__', '__package__', '__path__', '__version__', '_bsddb', '_checkflag', '_db', '_iter_mixin', '_openDBEnv', 'absolute_import', 'btopen', 'collections', 'db', 'dbutils', 'error', 'hashopen', 'os', 'ref', 'rnopen', 'sys']

Now you can copy _bsddb.so into your lib-dynload directory — e.g.:

sudo cp _bsddb.so /System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-dynload

(above is for the system Python 2.6 that comes with OS X)

or

sudo cp _bsddb.so sudo cp /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-dynload

(above is for a Python 2.6 installed from a .dmg from Python.org or built from source as a framework build)

References:

Make PyCrypto work with PyPy

I was trying to pip install PyCrypto in a PyPy virtualenv and kept getting build errors, mostly related to PyLongObject.

I worked around the problem by uninstalling the gmp library.

I’m on OS X so I did:

brew uninstall gmp

(If you’re on a different platform, replace “brew” with your package manager — apt-get, yum, etc.)

Then things worked:

(pypy-pycrypto)marca@marca-mac:~/python/virtualenvs$ pip install PyCrypto
Downloading/unpacking PyCrypto
  Real name of requirement PyCrypto is pycrypto
  Using download cache from /Users/marca/.pip/download-cache/https%3A%2F%2Fpypi.python.org%2Fpackages%2Fsource%2Fp%2Fpycrypto%2Fpycrypto-2.6.1.tar.gz
  Running setup.py egg_info for package PyCrypto

Installing collected packages: PyCrypto
...
Successfully installed PyCrypto
Cleaning up...

Voila!

Ag: The Silver Searcher

I don’t know if I’ve mentioned “ack” before — it’s been my favorite tool for searching code for a while (it’s better than grep, because it’s fairly speedy because it knows how to search only files that are code and how to ignore a lot of cruft plus it does nice output with coloring, paging, etc.).

Anyway, I just found an even better tool:

https://github.com/ggreer/the_silver_searcher

It’s more or less a drop-in replacement for “ack”; but it’s written in C (“ack” is Perl) and it’s faster. Looks like this:

Screen Shot of ag (the silver searcher)

Screen Shot of ag (the silver searcher)

And you can use it in your favorite editor probably:

https://github.com/leonid-shevtsov/SearchInProject_SublimeText2
https://github.com/rking/ag.vim
https://github.com/mileszs/ack.vim

Cython bug?

I have this Cython code:

cdef struct MyStruct:
    int x
 
def byte_bugginess(x=1):
    cdef char val = 1
    cdef MyStruct foo
 
    print("1 - val = %d" % val)
 
    if x in (3, 4):
        print("2 - val = %d" % val)
        val = 2
        print("3 - val = %d" % val)
    else:
        print("4 - val = %d" % val)
        val = foo.x
        print("5 - val = %d" % val)
 
    print("6 - val = %d" % val)

and the output is:

marca@marca-mac:~/dev/git-repos/cython-test$ make
python setup.py build_ext --inplace
running build_ext
gcc-4.2 not found, using clang instead
cythoning hello.pyx to hello.c
building 'hello' extension
creating build
creating build/temp.macosx-10.6-intel-2.7
clang -fno-strict-aliasing -fno-common -dynamic -isysroot /Developer/SDKs/MacOSX10.6.sdk -arch i386 -arch x86_64 -g -O2 -DNDEBUG -g -O3 -I/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c hello.c -o build/temp.macosx-10.6-intel-2.7/hello.o
clang -bundle -undefined dynamic_lookup -arch i386 -arch x86_64 -isysroot /Developer/SDKs/MacOSX10.6.sdk -isysroot /Developer/SDKs/MacOSX10.6.sdk -g build/temp.macosx-10.6-intel-2.7/hello.o -o /Users/marca/dev/git-repos/cython-test/hello.so
python -c 'import hello; hello.byte_bugginess(1)'
1 - val = 1
4 - val = 1
5 - val = 0
6 - val = 2

How could val be equal to 2? This implies that the false clause of the if statement is getting evaluated?

Weird.

Full code is at this gist.

I posted about this to the cython-users Google group. I am curious to see what they say.