#167 Cheating at Kaggle and uWSGI in prod

Sponsored by Datadog: pythonbytes.fm/datadog

Special guest: Vicki Boykis: @vboykis

Michael #1: clize: Turn functions into command-line interfaces

• via Marcelo
• Follow up from Typer on episode 164.
• Features
  • Create command-line interfaces by creating functions and passing them to [clize.run](https://clize.readthedocs.io/en/stable/api.html#clize.run).
  • Enjoy a CLI automatically created from your functions’ parameters.
  • Bring your users familiar --help messages generated from your docstrings.
  • Reuse functionality across multiple commands using decorators.
  • Extend Clize with new parameter behavior.
• I love how this is pure Python without its own API for the default case

Vicki #2: How to cheat at Kaggle AI contests

• Kaggle is a platform, now owned by Google, that allows data scientists to find data sets, learn data science, and participate in competitions
• Many people participate in Kaggle competitions to sharpen their data science/modeling skills
• Recently, a competition that was related to analyzing pet shelter data resulted in a huge controversy
• Petfinder.my is a platform that helps people find pets to rescue in Malaysia from shelters. In 2019, they announced a collaboration with Kaggle to create a machine learning predictor algorithm of which pets (worldwide) were more likely to be adopted based on the metadata of the descriptions on the site.
• The total prize offered was $25,000
• After several months, a contestant won. He was previously a Kaggle grandmaster, and won $10k.
• A volunteer, Benjamin Minixhofer, offered to put the algorithm in production, and when he did, he found that there was a huge discrepancy between first and second place
• Technical Aspects of the controversy:
  • The data they gave asked the contestants to predict the speed at which a pet would be adopted, from 1-5, and included input features like type of animal, breed, coloration, whether the animal was vaccinated, and adoption fee
  • The initial training set had 15k animals and the teams, after a couple months, were then given 4k animals that their algorithms had not seen before as a test of how accurate they were (common machine learning best practice).
  • In a Jupyter notebook Kernel on Kaggle, Minixhofer explains how the winning team cheated
  • First, they individually scraped Petfinder.my to find the answers for the 4k test data
  • Using md5, they created a hash for each unique pet, and looked up the score for each hash from the external dataset - there were 3500 overlaps
  • Did Pandas column manipulation to get at the hidden prediction variable for every 10th pet and replaces the prediction that should have been generated by the algorithm with the actual value
  • Using mostly: obfuscated functions, Pandas, and dictionaries, as well as MD5 hashes
• Fallout:
  • He was fired from H20.ai
  • Kaggle issued an apology

Michael #3: Configuring uWSGI for Production Deployment

• We run a lot of uWSGI backed services. I’ve spoken in-depth back on Talk Python 215: The software powering Talk Python courses and podcast about this.
• This is guidance from Bloomberg Engineering’s Structured Products Applications group
• We chose uWSGI as our host because of its performance and feature set. But, while powerful, uWSGI’s defaults are driven by backward compatibility and are not ideal for new deployments.
• There is also an official Things to Know doc.
• Unbit, the developer of uWSGI, has “decided to fix all of the bad defaults (especially for the Python plugin) in the 2.1 branch.” The 2.1 branch is not released yet.
• Warning, I had trouble with die-on-term and systemctl
• Settings I’m using:

# This option tells uWSGI to fail to start if any parameter
# in the configuration file isn’t explicitly understood by uWSGI.
strict = true

# The master uWSGI process is necessary to gracefully re-spawn
# and pre-fork workers, consolidate logs, and manage many other features
master = true

# uWSGI disables Python threads by default, as described in the Things to Know doc.
enable-threads = true

# This option will instruct uWSGI to clean up any temporary files or UNIX sockets it created
vacuum = true

# By default, uWSGI starts in multiple interpreter mode
single-interpreter = true

# Prevents uWSGI from starting if it is unable to find or load your application module
need-app = true

# uWSGI provides some functionality which can help identify the workers
auto-procname = true
procname-prefix = pythonbytes-

# Forcefully kill workers after 60 seconds. Without this feature,
# a stuck process could stay stuck forever.
harakiri = 60
harakiri-verbose = true

Vicki #4: Thinc: A functional take on deep learning, compatible with Tensorflow, PyTorch, and MXNet

• A deep learning library that abstracts away some TF and Pytorch boilerplate, from Explosion
• Already runs under the covers in SpaCy, an NLP library used for deep learning
• type checking, particularly helpful for Tensors: PyTorchWrapper and TensorFlowWrapper classes and the intermingling of both
• Deep support for numpy structures and semantics
• Assumes you’re going to be using stochastic gradient descent
• And operates in batches
• Also cleans up the configuration and hyperparameters
• Mainly hopes to make it easier and more flexible to do matrix manipulations, using a codebase that already existed but was not customer-facing.
• Examples and code are all available in notebooks in the GitHub repo

Michael #5: pandas-vet

• via Jacob Deppen
• A plugin for Flake8 that checks pandas code
• Starting with pandas can be daunting.
• The usual internet help sites are littered with different ways to do the same thing and some features that the pandas docs themselves discourage live on in the API.
• Makes pandas a little more friendly for newcomers by taking some opinionated stances about pandas best practices.
• The idea to create a linter was sparked by Ania Kapuścińska's talk at PyCascades 2019, "Lint your code responsibly!"

Vicki #6: NumPy beginner documentation

• NumPy is the backbone of numerical computing in Python: Pandas (which I mentioned before), scikit-learn, Tensorflow, and Pytorch, all lean heavily if not directly depend on its core concepts, which include matrix operations through a data structure known as a NumPy array (which is different than a Python list) - ndarray
• Anne Bonner wrote up new documentation for NumPy that introduces these fundamental concepts to beginners coming to both Python and scientific computing
• Before, you went directly to the section about arrays and had to search through it find what you wanted. The new guide, which is very nice, includes a step-by-step on how arrays work, how to reshape them, and illustrated guides on basic array operations.

Extras:

Vicki

• I write a newsletter, Normcore Tech, about all things tech that I’m not seeing covered in the mainstream tech media. I’ve written before about machine learning, data for NLP, Elon Musk memes, and Nginx.
• There’s a free version that goes out once a week and paid subscribers get access to one more newsletter per week, but really it’s more about the idea of supporting in-depth writing about tech. vicki.substack.com

Michael:

• pip 20.0 Released - Default to doing a user install (as if --user was passed) when the main site-packages directory is not writeable and user site-packages are enabled, cache wheels built from Git requirements, and more.
• Homebrew: brew install [email protected]

Joke:

An SEO expert walks into a bar, bars, pub, public house, Irish pub, tavern, bartender, beer, liquor, wine, alcohol, spirits...