ADL Internship Day Two — Using an SQL Connector instead of MySQL-Python; Because Sometimes, “Ya Gotta Go With What Works™”
Three major things of note happened today: I discovered a delicious new soup recipe, I started tracking my error messages and thought processes on Evernote, and I got the green light to try a different way of getting MySQL and Python to talk to each other from my boss.
Yesterday, I wrote about the challenges of getting the MySQL-Python package to download on my Mac. I did triumph in getting it to download (finally-more on that later), but when I went to import the package in Python via the command line, I got this error:
ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/MySQLdb/_mysql.so, 2): Symbol not found: _mysql_real_escape_string_quoteReferenced from: /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/MySQLdb/_mysql.soExpected in: flat namespacein /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/MySQLdb/_mysql.so
‘OK’, I think to myself. Let’s Google the error! I’ve never had to fix an import error before!
The challenge was, many of the guideposts I saw on Stack Overflow were *way* outside my comprehension. There was a lot of jargon, a lot of references to compilers, a ton of advice on manually redirecting paths (with varying degrees of success), and many, many instances where it seemed I was getting *very* close to a much lower level on my machine than I had ever delved before.
So, I started to break it down. I ran BrewDoctor to get a breakdown of my issues, then started to tackle them one by one. Some things I tried (in no particular order…)
- I updated XCode.
- I ran commands to allow compilers to find OpenSSL.
- I looked into if having Anaconda and Python 2.7 on my machine at the same time would cause an issue.
- I linked and unlinked MySQL-connector-c multiple times, with multiple versions of MySQL, from 5.7 to the current 8.0.
- I checked in with colleagues to see that this would even work with the current MySQL (apparently it does).
- I even looked into my hidden .bashrc file for some configuration flags that may be causing the issue.
This is not an exhaustive list! I spent a couple of hours researching, breaking things down to smaller and smaller bits, taking a walk, rinse, repeat. Nothing seemed to be making any progress as it seems the issue for the error varies widely from person to person and it appears to be a low-level/compiler/path issue. Then, I came across this error, which was very similar to mine, along with an explanation of the error from the package docs themselves:
ImportError: dlopen(./_mysql.so, 2): Symbol not found: _sprintf$LDBLStub Referenced from: ./_mysql.so Expected in: dynamic lookup
“This is one from Mac OS X. It seems to have been a compiler mismatch, but this time between two different versions of GCC. It seems nearly every major release of GCC changes the ABI in some why, so linking code compiled with GCC-3.3 and GCC-4.0, for example, can be problematic.”
Ohhhhhkay. I have no idea how to make two different versions of GCC talk to each other, and I know that’s not a casual thing to learn in an afternoon. And by this point, I knew I needed a long break and to honestly, reset my brain so I could be willing to take another approach.
See, this strange thing happens with our minds. We get so hooked on a particular path, that we start to think the path we are taking is the *only path possible* to solve a particular problem. I had spent about 2.5 hours on this path so far, researching and breaking things into tinier and tinier bits, and was no closer to solving my problem. I hadn’t received any new error messages and was going deeper and deeper into the weeds. I needed to abandon ship because my overall governance in thinking had become extremely linear and connected to “I must solve this by going deeper and deeper into the OS”. Obviously, that's a terrible idea because a) I highly doubt that is the ONLY way on God’s green earth to import this package into Python, b) I have no idea how to do anything at a low level right now and c) to learn how to do that would be an *enormous* waste of time for what I actually have to get done. So obviously, this path wasn’t working. I’m sure there *is* a way to get this working going deep into the OS or some such other way, but for right now, it's becoming a time and energy waste.
So what did I do? I took a walk! I picked up a pretty leaf, I shivered in the crisp autumn air, I thought about how lucky I am to be alive and how grateful I am to be outside, and the many, many more important things in the world than importing a package into Python 2.7. I had a good chuckle at that thought. Then I made myself the aforementioned mushroom soup.
After that, my boss suggested I give a SQL Connector a whack. I had come across this solution during my research but had it in my head that I *had* to use this way. This was an untrue assumption and a lesson learned: if you find a different way, suggest it, give it a whack, and clear it with your team to make sure it won’t break absolutely everything.
I had it up and running in twenty minutes.
Here are some resources I used (and will continue to use as a springboard) on how to use a MySQL Python Connector INSTEAD of the MySQL-Python package, explanations of what is happening under the hood, tutorials, exercises, and a link to the connector from Oracle’s website.
Python MySQL Database Connectivity [Complete Guide]
In this article, I will demonstrate how to perform MySQL database connectivity in Python using simple 5 steps. We are…
Python Database Programming Exercise with Solution
In this Python database exercise, we will do data insertion, data retrieval, data update and data deletion From…
I’m curious to poke around more in the future as to why this didn’t work, and I will maybe (hard maybe) publish my findings around that. Comments and tips are always welcome too!
Lastly, there is no such thing as wasted time when working on a difficult programming challenge. I always learn something. I learned more about working in Python, debugging, using Homebrew, using MySQL, reading error messages and figuring out paths and computer science principles from *not* being able to get this up and running than easily I would have ever learned if I’d gotten it done in 20 minutes.
‘Till next time :)