{"id":467,"date":"2020-07-22T23:24:50","date_gmt":"2020-07-23T03:24:50","guid":{"rendered":"http:\/\/blog.uvm.edu\/tbplante\/?p=467"},"modified":"2021-05-31T16:06:31","modified_gmt":"2021-05-31T20:06:31","slug":"getting-python-to-work-with-stata-in-windows","status":"publish","type":"post","link":"https:\/\/blog.uvm.edu\/tbplante\/2020\/07\/22\/getting-python-to-work-with-stata-in-windows\/","title":{"rendered":"Getting Python and Jupyter to work with Stata in Windows"},"content":{"rendered":"\n

Note: this post was written prior to Stata 17, which now allows Python to control Stata and vice versa. In Stata 16, Python could not control Stata but Stata could control Python. Because of this functionality, there’s a more streamlined approach to getting Jupyter to play nicely with Stata 17, detailed here<\/a>. You’ll still need to install Python and Jupyter, so this post is still helpful. <\/em><\/p>\n\n\n\n

Stata 16 now integrates with Python. I’m pretty stoked about using some of the Python figure packages. Getting it up and running has been a bit of a challenge. Here’s how I got it to work.<\/p>\n\n\n\n

Of note, since I started this post, Stata’s blog has started a series on using Python, which you should check out here<\/a>. <\/p>\n\n\n\n

Install part 1: <\/h2>\n\n\n\n

Installing Anaconda (free for individuals, not for institutions)<\/h3>\n\n\n\n

Anaconda comes with many built-in statistical packages. The (free) individual version of Anaconda from here<\/a>. Just make sure to check the “set as path” button during the Anaconda install<\/strong><\/em>!<\/em><\/strong><\/p>\n\n\n\n

Installing the (universally free) traditional Python distribution<\/h3>\n\n\n\n

As of July 2020, Python apparently has two versions that are commonly used, the 2.x version and the 3.x version. The end-of-life of 2.x versions is this year, so I wouldn’t recommend using it (current highest version is 2.7). Instead, use the 3.x version, currently the 3.8 version<\/strong>. You can find it at the Windows Python Download Page<\/a>.<\/p>\n\n\n\n

Make sure to install the version matching your Stata install!<\/em><\/strong> Stata comes as 32 bit or 64 bit. In Stata, type –about<\/strong>– to see what version you have. You’ll see that mine is running the 64-bit version of Stata. If you have a relatively modern computer, you are probably running the 64-bit version of Stata<\/em><\/strong>. Windows can actually run either 32-bit or 64-bit versions if you have a 64-bit processor, so do yourself a favor and just check. Type -about- in Stata to confirm your version.<\/em><\/strong> <\/p>\n\n\n\n

\"\"<\/figure>
\n

<\/p>\n<\/div><\/div>\n\n\n\n

Make sure that you install the corresponding version of Python. The highlighted one (x86-64) is the 64 bit. The other one (x86) is the 32-bit version. For this example, since I have the 64-bit version of stata, I installed the x86-64, 64-bit version of Python.<\/p>\n\n\n\n

I had originally installed the 32-bit version of Python and Stata couldn’t load it. Installing the 64-bit version of Python solved that. There’s actually a big “download now” button on the main Python webpage that will give you the 32-bit version. Make sure to select the specific stable release in the picture above. <\/p>\n\n\n\n

For the love of Pete, check this PATH box when you install it. <\/h3>\n\n\n\n

PATH is a list of commands that can be run from the Windows command line, and where their relative program exists. <\/p>\n\n\n\n

\"\"<\/figure>
\n

<\/p>\n<\/div><\/div>\n\n\n\n

See this check box right here? Select it. If you don’t, you’ll have a heck of a time getting anything to run from the command line. This should be checked on default, I have no idea why it’s not. If you forgot to check this box, uninstall Python and reinstall it after checking this box. <\/p>\n\n\n\n

Also, notice that it says “64 bit” on the installer screen above. If it says “32 bit”, you probably downloaded the wrong version. Go back and try again!<\/p>\n\n\n\n

What the heck did I just install?<\/h3>\n\n\n\n
\"\"<\/figure>
\n

<\/p>\n<\/div><\/div>\n\n\n\n

There are two Python shell apps\/programs that came along with the default Python setup. IDLE<\/a> is a more user-friendly Python shell. It resembles the command line in Stata, but it has the syntax highlighting of the Do file editor. <\/p>\n\n\n\n

The app called “Python 3.8 (64-bit)” is the shell without any markup. If you want to play around with Python, I recommend using IDLE. <\/p>\n\n\n\n

Making your first program in IDLE<\/h3>\n\n\n\n

Anything you run in Python should be from a script, or a *.py file. Pop one open from within IDLE by hitting Ctrl+N. Enter the following:<\/p>\n\n\n\n

\"\"<\/figure>
\n

<\/p>\n<\/div><\/div>\n\n\n\n

print(\"Hello world!\")<\/code><\/pre>\n\n\n\n
Then save it and run it (by pressing F5) and you’ll get the hello world!<\/p>\n\n\n\n
How does that look in Stata?<\/h3>\n\n\n\n
Let’s do the same thing in a Stata do file. In order to open up the Python shell within Stata, you have to type –python– on its own line, your intended python code, then –end– on its own line. Here, I have entered:<\/p>\n\n\n\n
python\nprint(\"hello world!\")\nend<\/code><\/pre>\n\n\n\n
Then just hit ctrl+d or the run button to get it to work in Stata!<\/p>\n\n\n\n
\"\"<\/figure>
\n
<\/p>\n<\/div><\/div>\n\n\n\n
Install Part 2: How do I get the Pandas, Matplotlib, SciPy, Sklearn, and NumPy libraries installed in Python?<\/h2>\n\n\n\n
Note: Anaconda comes with all of these except sklearn. For below, just complete the sklearn step.<\/em><\/p>\n\n\n\n
Python by itself can do some stuff, but the heavy lifting for stats and visualization is from add-in libraries that aren’t included with the default Python and must be added in before doing much of anything else. (Note: Anaconda<\/a> does <\/em>come with those and is a Python installation geared towards science, but we’re doing the classic install here.) Installing these additional libraries can be done with the included pip<\/strong> library, which automates all downloads and installations. BUT pip it has to be called from the Windows command line<\/em><\/strong>, not in a Python shell<\/em><\/strong> (i.e., not in IDLE). You’ll know you’re in the shell if the line starts with this:<\/p>\n\n\n\n
>>><\/pre>\n\n\n\n
So if you type “pip install pandas” in the shell (after the “>>>”), you’ll get an error and you will not be getting pandas. <\/p>\n\n\n\n
To pop up the Windows command line, hit the Start button then type “cmd” to open it up. Or hit windows key+r and type “cmd” to open it up.  If you correctly checked the PATH checkbox in the install, you should get the version reported if you type the following in:<\/p>\n\n\n\n
python --version<\/code><\/pre>\n\n\n\n
\"\"<\/figure>
\n
<\/p>\n<\/div><\/div>\n\n\n\n
If you get some sort of error, it’s probably because you didn’t check the PATH box during the install. Uninstall Python then reinstall it and make for sure you check that stupid PATH box. <\/p>\n\n\n\n
A note about the Windows 10 command line: If you type “Python” and hit enter, Windows pops up the Windows store and tries to get you to install the version of Python that they host. This is by far the dumbest Windows feature ever, and I have seen BOB. So, avoid ever typing the word “python” in the command line<\/em><\/strong>. Instead, use the handy “py” command, which does everything you’ll need it to do. Py is the python launcher. <\/p>\n\n\n\n
To call pip, you want to type in “py” then “-m” then “pip” and its commands. the “-m” allows you to run library commands as a script. If it’s been a while since you updated pip, now’s a good time to update it in the command line to show how to call pip:<\/p>\n\n\n\n
py -m pip install --upgrade pip<\/code><\/pre>\n\n\n\n
Now let’s install the core packages for Python:<\/p>\n\n\n\n
py -m pip install pandas\npy -m pip install matplotlib\npy -m pip install numpy\npy -m pip install scipy\npy -m pip install sklearn<\/code><\/pre>\n\n\n\n
You’ll see a screen like this, and it’ll let<\/p>\n\n\n\n
\"\"<\/figure>
\n
<\/p>\n<\/div><\/div>\n\n\n\n
You only need to do this installation step once. <\/p>\n\n\n\n
How do I use Pandas, Matplotlib, NumPy, Scikit-learn (sklearn), and SciPy in Stata?<\/h2>\n\n\n\n
Once the libraries are installed, you can then integrate them into your scripts. Each time you want to use them, you need to import them so you can call them. The convention is to import these using common so you don’t have to type “pandas” over and over again, you can just use “pd”. Ditto for other libraries:<\/p>\n\n\n\n
python\nimport pandas as pd\nimport matplotlib.pyplot as plt\nimport numpy as np\nimport scipy as sp\nimport sklearn as sk\nend<\/code><\/pre>\n\n\n\n
Install part 3: How do I install Jupyter Lab and get it to work with Stata? (This is the updated version of Jupyter Notebook)<\/h2>\n\n\n\n
Note: <\/em> Jupyter lab comes installed with Anaconda, but node.js, npm, and stata_kernel still must be installed.<\/em><\/p>\n\n\n\n
Jupyter is a super popular way to cleanly complete analyses. Its origins were in Python, but it now works in R and Stata. Details on installing Jupyter are here<\/a>. Specific instructions for getting it to interface with Stata are here<\/a>. Here’s how I got it installed:<\/p>\n\n\n\n
First: install node.js (warning: this takes a long time and requires a reboot)<\/h3>\n\n\n\n
Make sure that you have the current version of Python installed! <\/em><\/strong>Node.js will install the current version and if your version is outdated, it’ll create a huge headache. <\/p>\n\n\n\n
First, install node.js. Download here<\/a>. I checked the box in the node.js install to also install additional software. At the end of the install, it pops open Windows powershell window and installed a bunch of stuff, including Python if you do not have the current version. It also installs Chocolatey and a few other things. NOW STOP HERE AND REBOOT YOUR COMPUTER.<\/em><\/strong> Come back when it restarts. <\/p>\n\n\n\n
Next: install other npm, jupyterlab, and Stata plugins (warning: this also takes a while)<\/h3>\n\n\n\n
First, it’s always good habit to make sure you have the newest version of pip!<\/p>\n\n\n\n
py -m pip install --upgrade pip<\/code><\/pre>\n\n\n\n
Then, type the following in the Windows command line:<\/p>\n\n\n\n
py -m pip install npm\npy -m pip install jupyterlab\npy -m pip install stata_kernel\npy -m stata_kernel.install\njupyter labextension install jupyterlab-stata-highlight<\/code><\/pre>\n\n\n\n
When later trying to run jupyter lab, I got an error saying “Exception: Jupyter command jupyter-lab<\/code> not found.” I updated pip then uninstalled and reinstalled npm, jupyterlab, and stata_kernel (just replace the word “install” with “uninstall” above) then installed everything again. This fixed things. <\/p>\n\n\n\n
Finally, you need to do a last step described under “windows specific steps” here<\/a> <\/em><\/strong>to get this to work in Windows. I found my Stata executable at “C:\\Program Files<\/strong>\\Stata16\\StataSE-64.exe”, not Program Files (x86). You can delete the new Stata desktop shortcut once you run it as an administrator one time. <\/p>\n\n\n\n
You might also need to configure Jupyter to work with your Stata install. Details are here<\/a>. I found the configuration file named .stata_kernel.conf sitting in this folder: C:\\Users\\MYUSERNAME\\.stata_kernel.conf<\/p>\n\n\n\n
In reviewing the configuration file, it seems to have correctly identified my Stata SE 16 setup. I changed the graph format from svg to png, but left the rest unchanged. <\/p>\n\n\n\n
Now that I have Jupyter Lab installed, how do I open and use it?<\/h2>\n\n\n\n
In Anaconda Navigator, just click the “Jupyter lab” button. For a traditional Python install, open the Windows command line, type:<\/p>\n\n\n\n
jupyter lab<\/code><\/pre>\n\n\n\n
It should open up your web browser to a Jupyter page. Keep the Windows Command line terminal open in the background<\/em><\/strong>. If you close it, Jupyter will cease to work. (Note: This isn’t true for anaconda if you open Jupyter from the GUI.) Click the “Stata” notebook button to start.<\/p>\n\n\n\n
\"\"<\/figure>\n\n\n\n
It’ll open up a scripting page for your code. At the bottom it should say “stata idle”. That’s how you know you set it up correctly. <\/p>\n\n\n\n
\"\"<\/figure>\n\n\n\n
Now you can use traditional Stata code!<\/p>\n\n\n\n
There are additional programs called “Magics” detailed here<\/a> that help Stata integrate more seamlessly with Jupyter. Each of these commands begins with a % symbol. There are specific ways to modify these commands.<\/p>\n\n\n\n