Story Squad Putting Me to the Test
-Prologue-
Story Squad is a creative and insightful application being developed to help parents manage screen time use by their children, while building their confidence in refining their reading, writing, drawing, and teamwork skills. I encourage parents to take a look, and sign up with email to request early access. I had the privilege to join the development team for a short stint to put my skills to the “test” — quite literally. The story of this data scientist is below.
So, here is my opportunity to shine. I was put into a team of web developers and data scientists, all of whom were anxious to get into the project which was laid before us. As a data scientist myself, I was privy to the chatter within our little group. Are we going to build an awesome neural network model? Will we be a key in a major machine learning breakthrough? Are we going to be the ones to build an AI that can win Jeopardy against IBM’s Watson? The excitement was building.
As it turns out, it was something that none of us expected. The task before us was to build a testing suite for a mountain of already written code to test the endpoints. Off to the races we went in our research.
The first big question is: What framework do we use? Python has its own unittest. Pytest looked like a great option, too. FastAPI’s TestClient was also one to consider. In fact, in the existing code, FastAPI’s TestClient was already imported — but no tests were written yet. I did my best to research all three frameworks to choose which might be best for the task at hand. I took a deep look into all three. Let’s take a look at our options, shall we?
Python’s own unittest <docs>: Also called “PyUnit”, unittest is a framework that supports automation — something excellent for a testing suite to have. Running just one command will run all tests. There are a number of options that can be used within the code to compare outputs such as assertEqual to ensure an output is equal to the expected output, and assertRaises to ensure that a specific output raises a specific error. Some drawbacks to this option is that it is a bit harder to write than the others (a good bit of boilerplate code), and — if you are a person that gets bugged by this — it uses camelCase rather than a more python-friendly snake_case as its naming method (as seen in “assertEqual” and the like).
FastAPI’s TestClient <docs>: The one thing that stands out about this framework is that it lives up to its name — it’s indeed fast. It has been shown to outperform consistently in the speed realm. It also happens to be the FastAPI framework used in the application we were working with — rather than others like Flask or Django, so it could be a great match. Whereas this would certainly fulfill our requirements, the scope is more limited than the other two options. Once the endpoints were tested, another framework would be more appropriate for the rest of the functions that didn’t necessarily touch an endpoint.
Pytest <docs>: Pytest is a versatile option. This choice would permit unit testing, API testing, and functional testing. Also, it is an easy code to work with. This framework has a number of plugins to aid in versatility as to what possibilities are available in the testing realm, including “pytest html” which can print html reports with a simple command. Who wouldn’t rather look over a webpage report than scrolling through the text output of a terminal? One drawback that could be considered with this choice is that it is one of a kind. Compatibility currently struggles with other frameworks, should another be used.
Verdict
With these options in mind, Pytest seemed like the clear winner. Though TestClient was already imported, we thought best to abandon it. I found a very informative video on the implementation of Pytest (as shown here) and shared it with the team. We all studied up and prepared for our meeting with the stakeholders. This is the moment we have been waiting for. The team was ready to get started. We presented the question, “Which testing framework would you like us to use?”
Curveball
“We would prefer that you use the built-in unittest client — unless you can convince us otherwise.” We smiled and nodded, and thought silently of our rebuttal. In my head, I was thinking, “because we’ve already geared up for that one — and, it’s easier to write,” but I didn’t say that. It sounded silly. It sounded like the easy route. My mind gave me a 404 error, so one of us said “OK,” and we moved forward. The tried-and-true unittest it was.
Down the Rabbit Hole (part I)
This project had four main endpoints to consider for the test. We divided them up equally among us. The endpoint I was responsible for was illustration submission — where a user would upload a photo to the application to be processed. Because of the way this application had been written, it was advisable by our predecessors to use Linux if on a Windows machine — by way of wsl. After installing Ubuntu, restarting my computer, and getting the repo cloned, all was looking good. Next step was setting up the environment. In short, that didn’t work. A solid waste of time was happening in my world as I Googled the error message, did what was suggested to correct it, to be greeted by another error along the way. This cycle repeated until I gave up and started over completely. I uninstalled everything. I reinstalled everything. The ancient technical support mantra was being answered by what I was doing.
“Have you turned it off and back on again?” -Tech Support Agents Everywhere
Same failure.
So, I cloned the repository into my Windows machine and environment. Smooth sailing until met with errors indicating file paths were incorrect when trying to deploy the app locally. Files were missing. This is where my awesome teammate came to the rescue. Adding a blank file, and changing how some of the paths within the application, and voila! , it’s running. She wrote out a solid readme.md file to drop in the root of the repo for future developers who may stumble. Now, on to writing tests for the endpoints. Nothing but smooth sailing ahead, surely.
Down the Rabbit Hole (part II)
“Smooth sailing” was not what I experienced. I dug into the existing mountain of code to discern the path of my endpoint. A diagram was built using Whimsical and I shared it within my team’s Trello board — a place for technologically advanced virtual Post-It notes organized on a virtual whiteboard to share ideas and progress.
I wrote my first test to the endpoint. All I wanted was a status code of 200 — to show that the application was hearing me okay. So, I head to http://127.0.0.1:8000/ in my browser, and the Swagger API dashboard comes up. I hit execute, and hope for the best. After all, what could go wrong?
Well, apparently this can go wrong. We have a bad checksum. After doing some digging, I learned that the problem was not with the checksum — but with the file. Placing the filename into the address bar of my browser shows “Error — No such bucket.” Changing the error message itself was now on my personal “todo list”, as the checksum was only bad because the file didn’t exist.
And here is where this rabbit hole begins. I inquire about the whereabouts of the AWS Bucket to the stakeholders. I get given new keys to the current AWS Buckets. I disappoint my family by spending a good bit of my personal time learning everything I can about AWS integration into the Python language, and learn a lot of stuff that I’m not going to need to use. I learned a lot about the use of Boto3 in a hurry, just before realizing the truth that would have saved me a bunch of time. Since this particular endpoint was the submission of an illustration, why does the endpoint care that it comes from an AWS bucket at all? I climb out of my rabbit hole and vow not to enter any more.
Back on Track
With the rabbit holes squarely in my rearview mirror, I head forward. My team has been awaiting my success, and have been working on getting the functions leading to their respective endpoints with swiftness. Finally, with a clear path ahead to the MVP (Minimum Viable Product) goal. I had the framework for my endpoint ready for actual, real coding. So, once again, I was off to the races.
The first thing I had to do was to clean up the imports. Since we weren’t using a few of the modules, the imports looked a lot cleaner than before. I formed the request to the API, and hoped for the best.
In the end, the overall endeavor was a complete success. I got my endpoint. My teammates got their functions. I know the sight of the image below might not fill you with as much joy as it did me. Please, take a look:
So, with this, we hand it off to the next team. We will share with them what we did, what we didn’t do, and what could be done. In the end, I didn’t change that pesky “bad checksum” error in the programming. But, with the progress that has been made, it will be an easy thing to pick up.
So, as I leave the Story Squad team, I leave it with satisfaction of a job well done. This was not easy. Though satisfied, this didn’t come error-free. Of course there were lessons to be learned.
Lessons Learned and Hindsight — a list
- It is vital to be well-prepared for stakeholder meetings. Have answers, questions, thoughts, and everything written down. Or at least at the front of your thoughts.
- Test driven development (TDD), though tedious, is a method that I may employ in the future in my own projects, as it is easier to test as you go along rather than pilfering through a mountain of code.
- It may be wise to keep a virtual machine of more operating systems at the ready. It can help to ensure you have the right environment at the start of a project.
Conclusion
Thank you for taking the time to read my story and struggles. Please take a look through the document for links embedded, as they will guide you to definitions, docs, and explanations. And, please, go check out Story Squad. Take it for a whirl. I wrote a story and drew a picture, and you can too. Happy writing!