Seeing info when changing virtualenvs with virtualenvwrapper

I wanted to see a bit about virtualenvs when I activate them with workon from virtualenvwrapper.

I added the following to $VIRTUALENVWRAPPER_HOOK_DIR/postactivate:

# This hook is run after every virtualenv is activated.

python -V
easy_install --version
pip --version

Here’s how it looks:

$ workon pip
Python 2.7.6
setuptools 3.6
pip 6.0.dev1 from /Users/marca/dev/git-repos/pip (python 2.7)

For more information on customizing with hooks, read the docs.

Pull request to allow supervisor to send arbitrary signals to processes

Let’s you do stuff like:

$ supervisorctl
cat:0                            RUNNING   pid 57305, uptime 0:00:07
cat:1                            RUNNING   pid 57304, uptime 0:00:07
cat:2                            RUNNING   pid 57307, uptime 0:00:07
cat:3                            RUNNING   pid 57306, uptime 0:00:07
cat:4                            RUNNING   pid 57308, uptime 0:00:07
dog:0                            RUNNING   pid 57300, uptime 0:00:07
dog:1                            RUNNING   pid 57299, uptime 0:00:07
dog:2                            RUNNING   pid 57302, uptime 0:00:07
dog:3                            RUNNING   pid 57301, uptime 0:00:07
dog:4                            RUNNING   pid 57303, uptime 0:00:07
supervisor> help signal
signal <signal name> <name>	      Signal a process
signal <signal name> <gname>:*        Signal all processes in a group
signal <signal name> <name> <name>    Signal multiple processes or groups
supervisor> signal 1 dog:3 dog:4
dog:3: signalled
dog:4: signalled
supervisor> signal HUP dog:3 dog:4
dog:3: signalled
dog:4: signalled
supervisor> signal HUP dog:*
dog:1: signalled
dog:0: signalled
dog:3: signalled
dog:2: signalled
dog:4: signalled
supervisor> signal USR1 dog:1 dog:2
dog:1: signalled
dog:2: signalled

Also, if you can’t wait for supervisor to support this, the mr.laforge package supplies a supervisor plugin that can be used to send signals to processes:

$ supervisorctl kill HUP nginx

That said, it would be nice to have this built into supervisor…

Easier way to build _bsddb for OS X

I blogged previously about how to build _bsddb for Python 2.6 by hacking the Python source code to accommodate changes in berkeley-db.

Here’s an easier way that doesn’t require hacking source code.

Install berkeley-db using Homebrew:

brew install berkeley-db

Install the bsddb3 module, pointing it at an installation of berkeley-db:

sudo \
    BERKELEYDB_DIR=/usr/local/Cellar/berkeley-db/5.3.28 \
    pip-2.6 install bsddb3

Simply copy it into the appropriate place as This is super hacky and will fail if the interfaces of these two ever change, but for now for my limited purpose, it seems to work:

cp \
    /Library/Python/2.6/site-packages/bsddb3/ \
    /Library/Frameworks/Python.framework/Versions/2.6/lib \


$ python2.6
Python 2.6.7 (r267:88850, Oct 11 2012, 20:15:00)
[GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import bsddb
>>> dir(bsddb)
['MutableMapping', '_DBWithCursor', '_DeadlockWrap', '__builtins__', '__doc__', '__file__', '__name__', '__package__', '__path__', '__version__', '_bsddb', '_checkflag', '_db', '_iter_mixin', '_openDBEnv', 'absolute_import', 'btopen', 'collections', 'db', 'dbutils', 'error', 'hashopen', 'os', 'ref', 'rnopen', 'sys']

Building _bsddb for Python 2.6 on OS X 10.8.5

It’s well-known that the Python that comes with OS X machines typically doesn’t have _bsddb.

I downloaded Python 2.6.8 and attempted to build it and it wouldn’t build the _bsddb module.

Here’s what I did to fix this.

I used Homebrew to install berkeley-db4. For me, this grabbed version 4.8.30.

brew install berkeley-db4

Edited Modules/_bsddb.c and replaced all instances of DB_XIDDATASIZE with DB_GID_SIZE. I removed lines that had DB_XA_CREATE.

Compiled and linked with:

gcc -fno-strict-aliasing -fno-common -dynamic -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/Cellar/berkeley-db4/4.8.30/include/ -I/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c /Users/marca/src/Python-2.6.8/Modules/_bsddb.c -o _bsddb.o
gcc -bundle -undefined dynamic_lookup _bsddb.o -L/Library/Frameworks/Python.framework/Versions/2.6/lib -L/usr/local/lib -ldb -o -Wl,-search_paths_first

This created the file Modules/

At this point, I can import if I run python in the Modules directory where I built

$ python2.6
Python 2.6.7 (r267:88850, Oct 11 2012, 20:15:00)
[GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import bsddb
>>> dir(bsddb)
['MutableMapping', '_DBWithCursor', '_DeadlockWrap', '__builtins__', '__doc__', '__file__', '__name__', '__package__', '__path__', '__version__', '_bsddb', '_checkflag', '_db', '_iter_mixin', '_openDBEnv', 'absolute_import', 'btopen', 'collections', 'db', 'dbutils', 'error', 'hashopen', 'os', 'ref', 'rnopen', 'sys']

Now you can copy into your lib-dynload directory — e.g.:

sudo cp /System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-dynload

(above is for the system Python 2.6 that comes with OS X)


sudo cp sudo cp /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-dynload

(above is for a Python 2.6 installed from a .dmg from or built from source as a framework build)


Make PyCrypto work with PyPy

I was trying to pip install PyCrypto in a PyPy virtualenv and kept getting build errors, mostly related to PyLongObject.

I worked around the problem by uninstalling the gmp library.

I’m on OS X so I did:

brew uninstall gmp

(If you’re on a different platform, replace “brew” with your package manager — apt-get, yum, etc.)

Then things worked:

(pypy-pycrypto)marca@marca-mac:~/python/virtualenvs$ pip install PyCrypto
Downloading/unpacking PyCrypto
  Real name of requirement PyCrypto is pycrypto
  Using download cache from /Users/marca/.pip/download-cache/
  Running egg_info for package PyCrypto

Installing collected packages: PyCrypto
Successfully installed PyCrypto
Cleaning up...


Ag: The Silver Searcher

I don’t know if I’ve mentioned “ack” before — it’s been my favorite tool for searching code for a while (it’s better than grep, because it’s fairly speedy because it knows how to search only files that are code and how to ignore a lot of cruft plus it does nice output with coloring, paging, etc.).

Anyway, I just found an even better tool:

It’s more or less a drop-in replacement for “ack”; but it’s written in C (“ack” is Perl) and it’s faster. Looks like this:

Screen Shot of ag (the silver searcher)

Screen Shot of ag (the silver searcher)

And you can use it in your favorite editor probably:

Cython bug?

I have this Cython code:

cdef struct MyStruct:
    int x
def byte_bugginess(x=1):
    cdef char val = 1
    cdef MyStruct foo
    print("1 - val = %d" % val)
    if x in (3, 4):
        print("2 - val = %d" % val)
        val = 2
        print("3 - val = %d" % val)
        print("4 - val = %d" % val)
        val = foo.x
        print("5 - val = %d" % val)
    print("6 - val = %d" % val)

and the output is:

marca@marca-mac:~/dev/git-repos/cython-test$ make
python build_ext --inplace
running build_ext
gcc-4.2 not found, using clang instead
cythoning hello.pyx to hello.c
building 'hello' extension
creating build
creating build/temp.macosx-10.6-intel-2.7
clang -fno-strict-aliasing -fno-common -dynamic -isysroot /Developer/SDKs/MacOSX10.6.sdk -arch i386 -arch x86_64 -g -O2 -DNDEBUG -g -O3 -I/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c hello.c -o build/temp.macosx-10.6-intel-2.7/hello.o
clang -bundle -undefined dynamic_lookup -arch i386 -arch x86_64 -isysroot /Developer/SDKs/MacOSX10.6.sdk -isysroot /Developer/SDKs/MacOSX10.6.sdk -g build/temp.macosx-10.6-intel-2.7/hello.o -o /Users/marca/dev/git-repos/cython-test/
python -c 'import hello; hello.byte_bugginess(1)'
1 - val = 1
4 - val = 1
5 - val = 0
6 - val = 2

How could val be equal to 2? This implies that the false clause of the if statement is getting evaluated?


Full code is at this gist.

I posted about this to the cython-users Google group. I am curious to see what they say.