Compiling FreeTDS on Windows

This is how I got FreeTDS to compile on Windows 7 Professional with Visual Studio 2008.

1. Install Visual Studio 2008 (a.k.a.: Visual Studio 9.0)
2. Download the FreeTDS 0.91 source tarball and extract it somewhere. I chose C:\Users\marca\src\freetds-0.91.
3. Open a Visual Studio 2008 Command Prompt.
4. Change to the directory with FreeTDS:

cd C:\Users\marca\src\freetds-0.91

(or whatever you chose)
5. Give this a shot — it will fail probably:

nmake -fNmakefile -nologo apps PLATFORM=win32 CONFIGURATION=debug

6. You will get a bunch of errors that look like this:

c:\users\marca\src\freetds-0.91\include\tds_sysdep_private.h(239) :
fatal error C1083: Cannot open include file: 'inttypes.h':
No such file or directory

7. Fix the inttypes.h error — edit win32/config.h and comment out this line (line 92 for me):

#define HAVE_INTTYPES_H 1

8. If you repeat the above nmake command, then you’ll get a new error:

c:\program files (x86)\microsoft visual studio 9.0\vc\include\stdio.h
(358) : error C3163: '_vsnprintf': attributes inconsistent 
with previous declaration
        c:\program files (x86)\microsoft visual studio 9.0\vc\
        include\stdio.h(350) : see declaration of '_vsnprintf'

9. Fix the _vsnprintf error by editing include/tds_sysdep_private.h and commenting out the second instance of this line (line 96 for me):

#define vsnprintf _vsnprintf

10. Repeat the nmake command and this time it should succeed and you will end up with a tsql.exe file.

C:\Users\marca\src\freetds-0.91>nmake ^
More? -fNmakefile -nologo apps ^
More? PLATFORM=win32 CONFIGURATION=debug
...
C:\Users\marca\src\freetds-0.91>dir src\apps\win32\debug\tsql.exe
 Volume in drive C has no label.
 Volume Serial Number is D0C5-FA7C

 Directory of C:\Users\marca\src\freetds-0.91\src\apps\win32\debug

07/18/2013  04:19 PM           847,872 tsql.exe
               1 File(s)        847,872 bytes
               0 Dir(s)  19,728,400,384 bytes free

11. Give it a spin:

C:\Users\marca\src\freetds-0.91>src\apps\win32\debug\tsql ^
More? -H <server> -p 1433 ^
More? -U <username> -P <password> -D <database>
...
1> SELECT @@version
2> GO

Microsoft SQL Server 2008 R2 (SP2) - 10.50.4279.0 (X64)
        Mar 26 2013 17:33:13
        Copyright (c) Microsoft Corporation
        Developer Edition (64-bit) on Windows NT 6.1  
       (Build 7601: Service Pack 1) (Hypervisor)

(1 row affected)

A few useful links:

Python context manager for redirected stdout and stderr

I was working on the setup.py for pymssql and was annoyed by the distutils.ccompiler.CCompiler.has_function function — because it checks whether a C function exists by creating a very simple C program and invoking the compiler and linker and the compiler and linker may emit warnings and errors if the function doesn’t exist and then this shows up in the output of python setup.py install, etc. so it can confuse users and make users think that something is wrong when there really isn’t. Here’s an example of what I’m talking about:

$ python setup.py install
/var/folders/nk/8f8f6wjn7v3b9cb2gjqf6vph0000gp/T/clock_gettimeZdbS70.c:1:1: warning: type specifier missing, defaults to 'int' [-Wimplicit-int]
main (int argc, char **argv) {
^~~~
/var/folders/nk/8f8f6wjn7v3b9cb2gjqf6vph0000gp/T/clock_gettimeZdbS70.c:2:5: warning: implicit declaration of function 'clock_gettime' is invalid in C99 [-Wimplicit-function-declaration]
    clock_gettime();
    ^
2 warnings generated.
ld: library not found for -lrt
clang: error: linker command failed with exit code 1 (use -v to see invocation)

I don’t want users to see linker errors when they install the package and think that something is wrong. This is perfectly normal that -lrt didn’t work — this is on Mac OS X and it doesn’t have an rt library and we just detected that so that we don’t try to use it. The user doesn’t need to see these errors though, as it could confuse them.

So I looked for a way to not have the compiler errors displayed on stderr.

I found:

It seemed silly to have to reimplement logic in distutils for compiling code when all we want to do is just suppress error output by redirecting stderr. I thought one could simply call the distutils stuff inside a context manager that redirects stderr.

This is what I came up with — a context manager called stdchannel_redirected:

Here’s how I use it:

    with stdchannel_redirected(sys.stderr, os.devnull):
        if compiler.has_function('clock_gettime', libraries=['rt']):
            libraries.append('rt')

I used this in the setup.py of pymssql — see this commit.

I suspect that this probably exists out there in some other form, so I tweeted about it. Let me know if you know a better way…

OS X Homebrew: Build FreeTDS from HEAD (gitorious)

Here’s a pull request I just submitted to homebrew to make it possible to build FreeTDS from master on its gitorious repo:

https://github.com/mxcl/homebrew/pull/21309

Example of using it:

$ brew install freetds --HEAD --universal
==> Cloning https://git.gitorious.org/freetds/freetds.git
Updating /Library/Caches/Homebrew/freetds--git
==> autoreconf -i
==> ./configure --prefix=/usr/local/Cellar/freetds/HEAD --with-openssl=/usr/bin --with-tdsver=7.1 --mandir=/usr/local/Cellar/freetds/HEAD/share/man
==> make
==> make install
🍺  /usr/local/Cellar/freetds/HEAD: 63 files, 3.6M, built in 67 seconds

$ brew test freetds --HEAD -v
Testing freetds
==> /usr/local/Cellar/freetds/HEAD/bin/tsql -C
/usr/local/Cellar/freetds/HEAD/bin/tsql -C
Compile-time settings (established with the "configure" script)
                            Version: freetds v0.92.dev.20130718
             freetds.conf directory: /usr/local/Cellar/freetds/HEAD/etc
     MS db-lib source compatibility: no
        Sybase binary compatibility: no
                      Thread safety: yes
                      iconv library: yes
                        TDS version: 7.1
                              iODBC: yes
                           unixodbc: no
              SSPI "trusted" logins: no
                           Kerberos: no

Pretty simple:

commit 3d786f23bc2ff5d73a22474f9256330a17f01e23
Author: Marc Abramowitz 
Date:   Thu Jul 18 09:10:33 2013 -0700

    freetds.rb: Enable support for building FreeTDS from HEAD (gitorious)

diff --git a/Library/Formula/freetds.rb b/Library/Formula/freetds.rb
index 0a700e6..362b411 100644
--- a/Library/Formula/freetds.rb
+++ b/Library/Formula/freetds.rb
@@ -5,10 +5,19 @@ class Freetds < Formula
   url 'http://mirrors.ibiblio.org/freetds/stable/freetds-0.91.tar.gz'
   sha1 '3ab06c8e208e82197dc25d09ae353d9f3be7db52'

+  head 'https://git.gitorious.org/freetds/freetds.git'
+
   depends_on "pkg-config" => :build
   depends_on "unixodbc" => :optional

+  if build.head?
+    depends_on :automake
+    depends_on :libtool
+  end
+
   def install
+    system "autoreconf -i" if build.head?
+
     args = %W[--prefix=#{prefix}
               --with-openssl=/usr/bin
               --with-tdsver=7.1

Here’s the commit on GitHub.

I also added support for a test and for doing a universal build.

Compiling a universal library

A note for my future self on how to compile a library as universal binary in Mac OS X.

For a while, I’ve been getting warnings like these when building pymssql:

ld: warning: ignoring file /usr/local/lib/libsybdb.dylib, 
file was built for unsupported file format 
( 0xcf 0xfa 0xed 0xfe 0x 7 0x 0 0x 0 0x 1 0x 3 0x 0 0x 0 0x 0 0x 6 0x 0 0x 0 0x 0 ) 
which is not the architecture being linked (i386): 
/usr/local/lib/libsybdb.dylib
ld: warning: ignoring file /usr/local/lib/libct.dylib, 
file was built for unsupported file format 
( 0xcf 0xfa 0xed 0xfe 0x 7 0x 0 0x 0 0x 1 0x 3 0x 0 0x 0 0x 0 0x 6 0x 0 0x 0 0x 0 ) 
which is not the architecture being linked (i386): 
/usr/local/lib/libct.dylib

I finally decided to do something about them. I rebuilt FreeTDS from source as follows:

./configure \
    CFLAGS="-arch i386 -arch x86_64" \
    CXXFLAGS="-arch i386 -arch x86_64" \
    LDFLAGS="-arch i386 -arch x86_64" \
    --disable-dependency-tracking
make
make install

Goodbye warnings!

This Stack Overflow post was helpful: Stack Overflow: How to compile universal libraries on Mac OS X?

Back to setuptools

Browsing PyPI yesterday, I was surprised to see a new version of setuptools. I almost thought it was a bug. For years, I’ve been drinking the distribute Kool-Aid

pip/distribute Public Service Announcement

and automatically using distribute instead of setuptools. Well, I guess that things have changed and now distribute has been merged into setuptools and distribute is deprecated and setuptools is the thing to use. It may take me a little while to unlearn my old habits and it might make explaining Python packaging a little bit more difficult (not that it was easy :-)).

Note to self: Don’t name my git branches “tests”

I have sometimes had a git branch called “tests” because I was adding tests or working on tests. The problem is that many projects have a “tests” directory so if you have a “test” branch, you get this:

$ git diff tests
fatal: ambiguous argument 'tests': both revision and filename
Use '--' to separate paths from revisions, like this:
'git  [...] -- [...]'

You can of course use git diff -- tests and it will work, but it’s easier to just not name a branch “tests” in the first place.

Continuation of mod_wsgi setup

My previous setup worked until I wanted to add another Django instance and then it didn’t work, because I had two WSGIScriptAlias entries under different URLs but it would launch whichever app it wanted to regardless of the URL. A little searching suggested that it had to do with mod_wsgi process pools. I eventually ended up with something like this:

WSGIPythonHome /www/python/pypi.venv
WSGIPythonPath /www/python/django
WSGIPassAuthorization On
WSGISocketPrefix /var/run/wsgi

<VirtualHost *:80>

    ...

    DocumentRoot /www/python

        WSGIDaemonProcess djangoplayground_wsgi user=apache python-path=/www/python/django
        WSGIScriptAlias /djangoplayground /www/python/django/djangoplayground/wsgi.py
        <Location /djangoplayground>
            WSGIProcessGroup djangoplayground_wsgi
        </Location>

        WSGIDaemonProcess djangopypi_wsgi user=apache python-path=/www/python/django
        WSGIScriptAlias /djangopypi /www/python/django/djangopypisite/wsgi.py
        <Location /djangopypi>
            WSGIProcessGroup djangopypi_wsgi
        </Location>
</VirtualHost>