Seeing info when changing virtualenvs with virtualenvwrapper

I wanted to see a bit about virtualenvs when I activate them with workon from virtualenvwrapper.

I added the following to $VIRTUALENVWRAPPER_HOOK_DIR/postactivate:

# This hook is run after every virtualenv is activated.

python -V
easy_install --version
pip --version

Here’s how it looks:

$ workon pip
Python 2.7.6
setuptools 3.6
pip 6.0.dev1 from /Users/marca/dev/git-repos/pip (python 2.7)

For more information on customizing with hooks, read the docs.

Pull request to allow supervisor to send arbitrary signals to processes

Let’s you do stuff like:

$ supervisorctl
cat:0                            RUNNING   pid 57305, uptime 0:00:07
cat:1                            RUNNING   pid 57304, uptime 0:00:07
cat:2                            RUNNING   pid 57307, uptime 0:00:07
cat:3                            RUNNING   pid 57306, uptime 0:00:07
cat:4                            RUNNING   pid 57308, uptime 0:00:07
dog:0                            RUNNING   pid 57300, uptime 0:00:07
dog:1                            RUNNING   pid 57299, uptime 0:00:07
dog:2                            RUNNING   pid 57302, uptime 0:00:07
dog:3                            RUNNING   pid 57301, uptime 0:00:07
dog:4                            RUNNING   pid 57303, uptime 0:00:07
supervisor> help signal
signal <signal name> <name>	      Signal a process
signal <signal name> <gname>:*        Signal all processes in a group
signal <signal name> <name> <name>    Signal multiple processes or groups
supervisor> signal 1 dog:3 dog:4
dog:3: signalled
dog:4: signalled
supervisor> signal HUP dog:3 dog:4
dog:3: signalled
dog:4: signalled
supervisor> signal HUP dog:*
dog:1: signalled
dog:0: signalled
dog:3: signalled
dog:2: signalled
dog:4: signalled
supervisor> signal USR1 dog:1 dog:2
dog:1: signalled
dog:2: signalled

Also, if you can’t wait for supervisor to support this, the mr.laforge package supplies a supervisor plugin that can be used to send signals to processes:

$ supervisorctl kill HUP nginx

That said, it would be nice to have this built into supervisor…

Easier way to build _bsddb for OS X

I blogged previously about how to build _bsddb for Python 2.6 by hacking the Python source code to accommodate changes in berkeley-db.

Here’s an easier way that doesn’t require hacking source code.

Install berkeley-db using Homebrew:

brew install berkeley-db

Install the bsddb3 module, pointing it at an installation of berkeley-db:

sudo \
    BERKELEYDB_DIR=/usr/local/Cellar/berkeley-db/5.3.28 \
    pip-2.6 install bsddb3

Simply copy it into the appropriate place as This is super hacky and will fail if the interfaces of these two ever change, but for now for my limited purpose, it seems to work:

cp \
    /Library/Python/2.6/site-packages/bsddb3/ \
    /Library/Frameworks/Python.framework/Versions/2.6/lib \


$ python2.6
Python 2.6.7 (r267:88850, Oct 11 2012, 20:15:00)
[GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import bsddb
>>> dir(bsddb)
['MutableMapping', '_DBWithCursor', '_DeadlockWrap', '__builtins__', '__doc__', '__file__', '__name__', '__package__', '__path__', '__version__', '_bsddb', '_checkflag', '_db', '_iter_mixin', '_openDBEnv', 'absolute_import', 'btopen', 'collections', 'db', 'dbutils', 'error', 'hashopen', 'os', 'ref', 'rnopen', 'sys']

Building _bsddb for Python 2.6 on OS X 10.8.5

It’s well-known that the Python that comes with OS X machines typically doesn’t have _bsddb.

I downloaded Python 2.6.8 and attempted to build it and it wouldn’t build the _bsddb module.

Here’s what I did to fix this.

I used Homebrew to install berkeley-db4. For me, this grabbed version 4.8.30.

brew install berkeley-db4

Edited Modules/_bsddb.c and replaced all instances of DB_XIDDATASIZE with DB_GID_SIZE. I removed lines that had DB_XA_CREATE.

Compiled and linked with:

gcc -fno-strict-aliasing -fno-common -dynamic -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/Cellar/berkeley-db4/4.8.30/include/ -I/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c /Users/marca/src/Python-2.6.8/Modules/_bsddb.c -o _bsddb.o
gcc -bundle -undefined dynamic_lookup _bsddb.o -L/Library/Frameworks/Python.framework/Versions/2.6/lib -L/usr/local/lib -ldb -o -Wl,-search_paths_first

This created the file Modules/

At this point, I can import if I run python in the Modules directory where I built

$ python2.6
Python 2.6.7 (r267:88850, Oct 11 2012, 20:15:00)
[GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import bsddb
>>> dir(bsddb)
['MutableMapping', '_DBWithCursor', '_DeadlockWrap', '__builtins__', '__doc__', '__file__', '__name__', '__package__', '__path__', '__version__', '_bsddb', '_checkflag', '_db', '_iter_mixin', '_openDBEnv', 'absolute_import', 'btopen', 'collections', 'db', 'dbutils', 'error', 'hashopen', 'os', 'ref', 'rnopen', 'sys']

Now you can copy into your lib-dynload directory — e.g.:

sudo cp /System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-dynload

(above is for the system Python 2.6 that comes with OS X)


sudo cp sudo cp /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-dynload

(above is for a Python 2.6 installed from a .dmg from or built from source as a framework build)


Make PyCrypto work with PyPy

I was trying to pip install PyCrypto in a PyPy virtualenv and kept getting build errors, mostly related to PyLongObject.

I worked around the problem by uninstalling the gmp library.

I’m on OS X so I did:

brew uninstall gmp

(If you’re on a different platform, replace “brew” with your package manager — apt-get, yum, etc.)

Then things worked:

(pypy-pycrypto)marca@marca-mac:~/python/virtualenvs$ pip install PyCrypto
Downloading/unpacking PyCrypto
  Real name of requirement PyCrypto is pycrypto
  Using download cache from /Users/marca/.pip/download-cache/
  Running egg_info for package PyCrypto

Installing collected packages: PyCrypto
Successfully installed PyCrypto
Cleaning up...


Cython bug?

I have this Cython code:

cdef struct MyStruct:
    int x
def byte_bugginess(x=1):
    cdef char val = 1
    cdef MyStruct foo
    print("1 - val = %d" % val)
    if x in (3, 4):
        print("2 - val = %d" % val)
        val = 2
        print("3 - val = %d" % val)
        print("4 - val = %d" % val)
        val = foo.x
        print("5 - val = %d" % val)
    print("6 - val = %d" % val)

and the output is:

marca@marca-mac:~/dev/git-repos/cython-test$ make
python build_ext --inplace
running build_ext
gcc-4.2 not found, using clang instead
cythoning hello.pyx to hello.c
building 'hello' extension
creating build
creating build/temp.macosx-10.6-intel-2.7
clang -fno-strict-aliasing -fno-common -dynamic -isysroot /Developer/SDKs/MacOSX10.6.sdk -arch i386 -arch x86_64 -g -O2 -DNDEBUG -g -O3 -I/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c hello.c -o build/temp.macosx-10.6-intel-2.7/hello.o
clang -bundle -undefined dynamic_lookup -arch i386 -arch x86_64 -isysroot /Developer/SDKs/MacOSX10.6.sdk -isysroot /Developer/SDKs/MacOSX10.6.sdk -g build/temp.macosx-10.6-intel-2.7/hello.o -o /Users/marca/dev/git-repos/cython-test/
python -c 'import hello; hello.byte_bugginess(1)'
1 - val = 1
4 - val = 1
5 - val = 0
6 - val = 2

How could val be equal to 2? This implies that the false clause of the if statement is getting evaluated?


Full code is at this gist.

I posted about this to the cython-users Google group. I am curious to see what they say.

Compiling FreeTDS on Windows

This is how I got FreeTDS to compile on Windows 7 Professional with Visual Studio 2008.

1. Install Visual Studio 2008 (a.k.a.: Visual Studio 9.0)
2. Download the FreeTDS 0.91 source tarball and extract it somewhere. I chose C:\Users\marca\src\freetds-0.91.
3. Open a Visual Studio 2008 Command Prompt.
4. Change to the directory with FreeTDS:

cd C:\Users\marca\src\freetds-0.91

(or whatever you chose)
5. Give this a shot — it will fail probably:

nmake -fNmakefile -nologo apps PLATFORM=win32 CONFIGURATION=debug

6. You will get a bunch of errors that look like this:

c:\users\marca\src\freetds-0.91\include\tds_sysdep_private.h(239) :
fatal error C1083: Cannot open include file: 'inttypes.h':
No such file or directory

7. Fix the inttypes.h error — edit win32/config.h and comment out this line (line 92 for me):


8. If you repeat the above nmake command, then you’ll get a new error:

c:\program files (x86)\microsoft visual studio 9.0\vc\include\stdio.h
(358) : error C3163: '_vsnprintf': attributes inconsistent 
with previous declaration
        c:\program files (x86)\microsoft visual studio 9.0\vc\
        include\stdio.h(350) : see declaration of '_vsnprintf'

9. Fix the _vsnprintf error by editing include/tds_sysdep_private.h and commenting out the second instance of this line (line 96 for me):

#define vsnprintf _vsnprintf

10. Repeat the nmake command and this time it should succeed and you will end up with a tsql.exe file.

C:\Users\marca\src\freetds-0.91>nmake ^
More? -fNmakefile -nologo apps ^
C:\Users\marca\src\freetds-0.91>dir src\apps\win32\debug\tsql.exe
 Volume in drive C has no label.
 Volume Serial Number is D0C5-FA7C

 Directory of C:\Users\marca\src\freetds-0.91\src\apps\win32\debug

07/18/2013  04:19 PM           847,872 tsql.exe
               1 File(s)        847,872 bytes
               0 Dir(s)  19,728,400,384 bytes free

11. Give it a spin:

C:\Users\marca\src\freetds-0.91>src\apps\win32\debug\tsql ^
More? -H <server> -p 1433 ^
More? -U <username> -P <password> -D <database>
1> SELECT @@version
2> GO

Microsoft SQL Server 2008 R2 (SP2) - 10.50.4279.0 (X64)
        Mar 26 2013 17:33:13
        Copyright (c) Microsoft Corporation
        Developer Edition (64-bit) on Windows NT 6.1  
       (Build 7601: Service Pack 1) (Hypervisor)

(1 row affected)

A few useful links: