Compiling Python 2.7 Modules on Windows 32 and 64 using MSVC++ 2008 Express
05/06/2011 22 Comments
Hello,
On this post i will explain how to build, compile, install and distribute python modules on Windows using Microsoft Visual C++ Express Edition.
This post will be constantly updated to cover future updates of python, windows and msvc++ versions.
For this example i will use the PyCrypto – http://pycrypto.org because this is an example that don’t have packages for windows x64 on the web.
Observation: Don’t use Microsoft Visual C++ Express Edition 2010 to build python modules, because this will not work due to Python 2.7 was built using the 2008 version. This is an error that occurs when you try to build PyCrypto and Paramiko using the 2010 version and execute the import module:
>>> import paramiko
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python27\lib\site-packages\paramiko\__init__.py", line 69, in <module>
from transport import SecurityOptions, Transport
File "C:\Python27\lib\site-packages\paramiko\transport.py", line 32, in <module>
from paramiko import util
File "C:\Python27\lib\site-packages\paramiko\util.py", line 32, in <module>
from paramiko.common import *
File "C:\Python27\lib\site-packages\paramiko\common.py", line 98, in <module>
from Crypto import Random
File "C:\Python27\lib\site-packages\Crypto\Random\__init__.py", line 28, in <module>
import OSRNG
File "C:\Python27\lib\site-packages\Crypto\Random\OSRNG\__init__.py", line 34, in <module>
from Crypto.Random.OSRNG.nt import new
File "C:\Python27\lib\site-packages\Crypto\Random\OSRNG\nt.py", line 28, in <module>
import winrandom
ImportError: DLL load failed: The specified module could not be found.
1 – Building and Installing PyCrypto Module for Windows 7 64 bits:
1.1 - You must have installed the Python 64 bits version:
http://www.python.org/ftp/python/2.7.1/python-2.7.1.amd64.msi
1.2 - You should install the C Compiler for Windows – Microsoft Visual C++ Express Edition 2008: available here:
http://www.microsoft.com/express/Downloads/#Visual_Studio_2008_Express_Downloads
ISO File to Download:
http://www.microsoft.com/express/Downloads/#2008-All
1.3 - You should install the Microsoft Windows SDK for Windows 7 and .NET Framework 3.5 SP1: available here:
http://www.microsoft.com/downloads/en/details.aspx?FamilyID=c17ba869-9671-4330-a63e-1fd44e0e2505
ISO File To Download (64 bits):
http://download.microsoft.com/download/2/E/9/2E911956-F90F-4BFB-8231-E292A7B6F287/GRMSDKX_EN_DVD.iso
This is required because the Express Edition 2008 C++ don’t contains the 64 bits compiler. This is required only for Windows 7 64 bits version.
Important: Don’t use the “Microsoft Windows SDK for Windows 7 and .NET Framework 4″ because it’s not compatible with msvc++ express 2008 edition.
1.4 - Install the Python Setup Tools available here:
http://pypi.python.org/pypi/setuptools#downloads
1.5 - Include in your Advanced Variables Environment the binaries of Python. Right click at “My Computer” icon -> Properties -> Advanced Environment and edit your Path Variable including this two directories there:
Path = C:\Python27\Scripts;C:\Python27; + Path
1) C:\Python27\Scripts 2) C:\Python27
1.6 - Copy this file:
C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\bin\vcvars64.bat
To this follow folder and rename the file (vcvars64.bat to vcvarsamd64.bat):
C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\bin\amd64\vcvarsamd64.bat
1.7 - Edit the file msvc9compiler.py inside of directory C:\Python27\Lib\distutils\msvc9compiler.py
After the line 651 approximately, find this line: – ld_args.append(‘/MANIFESTFILE:’ + temp_manifest)
Add the following line after the above line:
ld_args.append('/MANIFEST')
1.8 - Edit the file msvccompiler.py inside of directory C:\Python27\Lib\distutils\msvccompiler.py
At line 153 approximately, insert this line: return 9.0, as following, in this piece of code:
def get_build_version():
"""Return the version of MSVC that was used to build Python.
For Python 2.3 and up, the version number is included in
sys.version. For earlier versions, assume the compiler is MSVC 6.
"""
return 9.0
prefix = "MSC v."
i = string.find(sys.version, prefix)
if i == -1:
return 6
1.9 - Certify that exists the follow environment variable in your system, if don’t exist create a new one:
Name: VS90COMNTOOLS Value: C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\Tools\
1.10 - Extract the file pycrypto-2.3.tar.gz that you downloaded before, open a dos command and access the pycrypto-2.3 folder
1.11 - Execute this command to compile, build and install the module:
python setup.py build --compiler msvc python setup.py install
1.12 - Test if the package was generated successfully executing an import Crypto at Python shell.
1.13 - It’s finish. Optionally, you can distribute your compiled modules as an windows executable (.exe) file, executing this simple command on folder of your module, this case is pycrypto-2.3:
python setup.py bdist_wininst
As a result you will get a executable file with graphical interface created inside of folder “dist” and other people can install this executable file without to prepare a complex environment for build your own module. This is my result file of this process.
http://arquivos.victorjabur.com/python/modules/pycrypto-2.3.win-amd64-py2.7.exe
2 – Building and Installing PyCrypto Module for Windows All Versions 32 bits:
2.1 - You must have installed the Python 32 bits version:
http://www.python.org/ftp/python/2.7.1/python-2.7.1.msi
2.2 - You should install the C Compiler for Windows – Microsoft Visual C++ Express Edition 2008: available here:
http://www.microsoft.com/express/Downloads/#Visual_Studio_2008_Express_Downloads
ISO File to Download:
http://www.microsoft.com/express/Downloads/#2008-All
2.3 - Install the Python Setup Tools available here:
http://pypi.python.org/pypi/setuptools#downloads
2.4 - Include in your Advanced Variables Environment the binaries of Python. Right click at “My Computer” icon -> Properties -> Advanced Environment and edit your Path Variable including this two directories there:
Path = C:\Python27\Scripts;C:\Python27; + Path
1) C:\Python27\Scripts 2) C:\Python27
2.5 - Edit the file msvc9compiler.py inside of directory C:\Python27\Lib\distutils\msvc9compiler.py
After the line 651 approximately, find this line: – ld_args.append(‘/MANIFESTFILE:’ + temp_manifest)
Add the following line after the above line:
ld_args.append('/MANIFEST')
2.6 - Edit the file msvccompiler.py inside of directory C:\Python27\Lib\distutils\msvccompiler.py
At line 153 approximately, insert this line: return 9.0, as following, in this piece of code:
def get_build_version():
"""Return the version of MSVC that was used to build Python.
For Python 2.3 and up, the version number is included in
sys.version. For earlier versions, assume the compiler is MSVC 6.
"""
return 9.0
prefix = "MSC v."
i = string.find(sys.version, prefix)
if i == -1:
return 6
2.7 - Certify that exists the follow environment variable in your system, if don’t exist create a new one:
Name: VS90COMNTOOLS Value: C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\Tools\
2.8 - Extract the file pycrypto-2.3.tar.gz that you downloaded before, open a dos command and access the pycrypto-2.3 folder
2.9 - Execute this command to compile, build and install the module:
python setup.py build --compiler msvc python setup.py install
2.10 - Test if the package was generated successfully executing an import Crypto at Python shell.
2.11 - It’s finish. Optionally, you can distribute your compiled modules as an windows executable (.exe) file, executing this simple command on folder of your module, this case is pycrypto-2.3:
python setup.py bdist_wininst
As a result you will get a executable file with graphical interface created inside of folder “dist” and other people can install this executable file without to prepare a complex environment for build your own module. This is my result file of this process.
http://arquivos.victorjabur.com/python/modules/pycrypto-2.3.win32-py2.7.exe
Credits and References to this post:
http://nukeit.org/compile-python-2-7-packages-with-visual-studio-2010-express/
http://yorickdowne.wordpress.com/2010/12/22/compiling-pycrypto-on-win7-64/
http://www.lfd.uci.edu/~gohlke/pythonlibs/
http://www.fuyun.org/2009/12/install-mysql-for-python-on-windows/
http://mattptr.net/2010/07/28/building-python-extensions-in-a-modern-windows-environment/
Good Bye.
Victor Jabur
Hi Victor,
thanks for the helpful blog, I tried to follow a few of your suggestions, but ran into some issues. I tried to compile the mpi4py package on windows, and got the following error:
C:\TEMP\mpi4py-1.2.2>python setup.py build –compiler msvc
running build
running build_py
running build_ext
MPI configuration: directory ‘C:\Program Files\MPICH2′
MPI C compiler: not found
MPI C++ compiler: not found
MPI linker: not found
checking for MPI compile and link …
Traceback (most recent call last):
File “setup.py”, line 316, in
main()
File “setup.py”, line 306, in main
run_setup()
File “setup.py”, line 279, in run_setup
**metadata)
File “C:\TEMP\mpi4py-1.2.2\conf\mpidistutils.py”, line 694, in setup
return fcn_setup(**attrs)
File “C:\Python27\lib\distutils\core.py”, line 152, in setup
dist.run_commands()
File “C:\Python27\lib\distutils\dist.py”, line 953, in run_commands
self.run_command(cmd)
File “C:\Python27\lib\distutils\dist.py”, line 972, in run_command
cmd_obj.run()
File “C:\Python27\lib\distutils\command\build.py”, line 127, in run
self.run_command(cmd_name)
File “C:\Python27\lib\distutils\cmd.py”, line 326, in run_command
self.distribution.run_command(command)
File “C:\Python27\lib\distutils\dist.py”, line 972, in run_command
cmd_obj.run()
File “C:\Python27\lib\distutils\command\build_ext.py”, line 340, in run
self.build_extensions()
File “C:\TEMP\mpi4py-1.2.2\conf\mpidistutils.py”, line 1188, in build_extensio
ns
self.config_extension(ext, config_info)
File “C:\TEMP\mpi4py-1.2.2\conf\mpidistutils.py”, line 1215, in config_extensi
on
headers=['stdlib.h', 'mpi.h'])
File “C:\Python27\lib\distutils\command\config.py”, line 251, in try_link
libraries, library_dirs, lang)
File “C:\Python27\lib\distutils\command\config.py”, line 143, in _link
(src, obj) = self._compile(body, headers, include_dirs, lang)
File “C:\Python27\lib\distutils\command\config.py”, line 138, in _compile
self.compiler.compile([src], include_dirs=include_dirs)
File “C:\Python27\lib\distutils\msvc9compiler.py”, line 478, in compile
self.initialize()
File “C:\Python27\lib\distutils\msvc9compiler.py”, line 388, in initialize
vc_env = query_vcvarsall(VERSION, plat_spec)
File “C:\Python27\lib\distutils\msvc9compiler.py”, line 304, in query_vcvarsal
l
raise ValueError(str(list(result.keys())))
ValueError: [u'path']
C:\TEMP\mpi4py-1.2.2>
After digging into query_vcvarsal it seems that the problem is that after looking through the path variables, the python cannot find any entries for:
interesting = set((“include”, “lib”, “libpath”))
Do I need to manually create more variables?
Hi Federico V.
It works for me, you could have an environment problem, check if you follow all steps on the post.
Here is the commands that i used to build mpi4py:
Main Page: http://www.mcs.anl.gov/research/projects/mpich2/downloads/index.php?s=downloads
Download of MPICH2 for Windows 64 bits: http://www.mcs.anl.gov/research/projects/mpich2/downloads/tarballs/1.3.2p1/mpich2-1.3.2p1-win-x86-64.msi
python setup.py build –compiler msvc –mpicc=”C:\Program Files\MPICH2\bin”
Download of MPICH2 for Windows 32 bits: http://www.mcs.anl.gov/research/projects/mpich2/downloads/tarballs/1.3.2p1/mpich2-1.3.2p1-win-ia32.msi
python27_32 setup.py build –compiler msvc –mpicc=”C:\Program Files (x86)\MPICH2\bin”
Here is the compiled files:
For Windows 64 bits: http://arquivos.victorjabur.com/python/modules/mpi4py-1.2.2.win-amd64-py2.7.exe
For Windows 32 bits: http://arquivos.victorjabur.com/python/modules/mpi4py-1.2.2.win32-py2.7.exe
Pingback: Compiling MySQLdb 1.2.3 on Windows 32 and 64 – Without need the MySQL Database 5.5 installed « Victor Jabur's Blog
Hi Victor,
do you have an email address where I could contact you further? I followed all the steps but I think there is a path variable that is missing somewhere.
Thank you very much for the pre-compiled file though,
Federico
victorjabur@gmail.com
When I follow the steps above for 64-bit Windows 7 I get the following error:
running build
running build_py
running build_ext
warning: GMP library not found; Not building Crypto.PublicKey._fastmath.
building ‘Crypto.Random.OSRNG.winrandom’ extension
Traceback (most recent call last):
File “setup.py”, line 340, in
core.setup(**kw)
File “C:\Python27\lib\distutils\core.py”, line 152, in setup
dist.run_commands()
File “C:\Python27\lib\distutils\dist.py”, line 953, in run_commands
self.run_command(cmd)
File “C:\Python27\lib\distutils\dist.py”, line 972, in run_command
cmd_obj.run()
File “C:\Python27\lib\distutils\command\build.py”, line 127, in run
self.run_command(cmd_name)
File “C:\Python27\lib\distutils\cmd.py”, line 326, in run_command
self.distribution.run_command(command)
File “C:\Python27\lib\distutils\dist.py”, line 972, in run_command
cmd_obj.run()
File “C:\Python27\lib\distutils\command\build_ext.py”, line 340, in run
self.build_extensions()
File “setup.py”, line 152, in build_extensions
build_ext.build_extensions(self)
File “C:\Python27\lib\distutils\command\build_ext.py”, line 449, in build_extensions
self.build_extension(ext)
File “C:\Python27\lib\distutils\command\build_ext.py”, line 499, in build_extension
depends=ext.depends)
File “C:\Python27\lib\distutils\msvc9compiler.py”, line 473, in compile
self.initialize()
File “C:\Python27\lib\distutils\msvc9compiler.py”, line 383, in initialize
vc_env = query_vcvarsall(VERSION, plat_spec)
File “C:\Python27\lib\distutils\msvc9compiler.py”, line 299, in query_vcvarsall
raise ValueError(str(list(result.keys())))
ValueError: [u'path']
Any thoughts?
Check if you have copied bin\vcvars64.bat into bin\amd64 and renamed it vcvarsamd64.bat
I had the same error message and renaming was the step I forgot to do.
Thanks man, I’ve been hacking around trying to figure this one out for a while. It worked like a champ.
Hi Victor,
thank you for your helpful blog entry.
It helped me to compile PIL on Windows 7 64 bit.
Hi, thanks for your precise instructions that helped me compile pycrypto on 64-bit. It was a frustrating process with all the other instructions on the Internet. Your blog was a night and day difference. Keep up the good work! Excellent!
Thanks a lot for this!
Thank you very much! This helped me compile pybluez for python 2.7 on W7. Thanks!
Dewald, can you share your binaries for pybluez?
32 or 64 bits ?
On this page exists binaries, http://code.google.com/p/pybluez/downloads/list what version you want ?
Hello Dewald, hellow Alexander, hello Victor,
can you please share the Win 7 PyBluez binaries for x32 , Python 2.7 ?
Thanks in advance!
Thanks Victor for your very precise tutorial. It saved me a lot of time and frustration when installing python packages that requires compilation. Keep up the good work!
Pingback: Mercurial on IIS 7 – x86 or x64 | pteradigm
i also tried following the above steps for installing rpy2 in windows7 64 bit but it is giving below error message. Victor it would be nice if you can check:
C:\Program Files\Microsoft SDKs\Windows\v7.0\include\ws2def.h(961) : warning C4068: unknown pragma
c:\users\art\desktop\start\rpy2\rpy2-2.3.1\rpy2-2.3.1\rpy\rinterface\embeddedr.h(6) : warning C4114: same type qualifier used more than once
c:\users\art\desktop\start\rpy2\rpy2-2.3.1\rpy2-2.3.1\rpy\rinterface\embeddedr.h(7) : warning C4114: same type qualifier used more than once
c:\users\art\desktop\start\rpy2\rpy2-2.3.1\rpy2-2.3.1\rpy\rinterface\embeddedr.h(22) : error C2054: expected ‘(‘ to follow ‘inline’
c:\users\art\desktop\start\rpy2\rpy2-2.3.1\rpy2-2.3.1\rpy\rinterface\embeddedr.h(22) : error C2085: ‘Rpy_ReplaceSexp’ : not in formal parameter list
c:\users\art\desktop\start\rpy2\rpy2-2.3.1\rpy2-2.3.1\rpy\rinterface\na_values.h(8) : error C2085: ‘NAInteger_Type’ : not in formal parameter list
c:\users\art\desktop\start\rpy2\rpy2-2.3.1\rpy2-2.3.1\rpy\rinterface\na_values.h(9) : error C2085: ‘NAReal_Type’ : not in formal parameter list
c:\users\art\desktop\start\rpy2\rpy2-2.3.1\rpy2-2.3.1\rpy\rinterface\na_values.h(10) : error C2085: ‘NAComplex_Type’ : not in formal parameter list
c:\users\art\desktop\start\rpy2\rpy2-2.3.1\rpy2-2.3.1\rpy\rinterface\na_values.h(11) : error C2085: ‘NALogical_Type’ : not in formal parameter list
c:\users\art\desktop\start\rpy2\rpy2-2.3.1\rpy2-2.3.1\rpy\rinterface\na_values.h(12) : error C2085: ‘NACharacter_Type’ : not in formal parameter list
c:\users\art\desktop\start\rpy2\rpy2-2.3.1\rpy2-2.3.1\rpy\rinterface\na_values.h(17) : error C2057: expected constant expression
c:\users\art\desktop\start\rpy2\rpy2-2.3.1\rpy2-2.3.1\rpy\rinterface\na_values.h(18) : error C2085: ‘ieee_double’ : not in formal parameter list
c:\users\art\desktop\start\rpy2\rpy2-2.3.1\rpy2-2.3.1\rpy\rinterface\na_values.h(23) : error C2085: ‘ieee_double’ : not in formal parameter list
error: command ‘”C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\amd64\cl.exe”‘ failed with exit status 2
I will verify
I am trying to compile pylibdnet for Windows because the last Windows installer for libdnet was published in 2009 for Python 2.4. I have followed your instructions for 32 bit Windows and I obtain the error message:
Could not locate the include file “libnet.h”
There is not an include folder with my pylibnet package where I might expect to find libnet.h, but there is also not an include folder with your pycrypto-2.6 package, and yet you compiled it successfully. Can you explain what I am doing wrong?
I also have an up to date version of libnet for developers and I wondered if someone could advise me on how to compile that from source bearing in mind that I am a clueless Windows user.
Pingback: GAE Python - PyCrypto - No module named winrandom | BlogoSfera