Exploring Literate Programming For Python Projects With nbdev

Summary Creating well designed software is largely a problem of context and understanding. The majority of programming environments rely on documentation, tests, and code being logically separated despite being contextually linked. In order to weave all of these concerns together there have been many efforts to create a literate programming environment. In this episode Jeremy Howard of fast.ai fame and Hamel Husain of GitHub share the work they have done on nbdev. The explain how it allows you to weave together documentation, code, and tests in the same context so that it is more natural to explore and build understanding when working on a project. It is built on top of the Jupyter environment, allowing you to take advantage of the other great elements of that ecosystem, and it provides a number of excellent out of the box features to reduce the friction in adopting good project hygiene, including continuous integration and well designed documentation sites. Regardless of whether you have been programming for 5 days, 5 years, or 5 decades you should take a look at nbdev to experience a different way of looking at your code. Announcements Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great. When you’re ready to launch your next app or want to try a project you hear about on the show, you’ll need somewhere to deploy it, so take a look at our friends over at Linode. With the launch of their managed Kubernetes platform it’s easy to get started with the next generation of deployment and scaling, powered by the battle tested Linode platform, including simple pricing, node balancers, 40Gbit networking, dedicated CPU and GPU instances, and worldwide data centers. Go to pythonpodcast.com/linode and get a $100 credit to try out a Kubernetes cluster of your own. And don’t forget to thank them for their continued support of this show! Your host as usual is Tobias Macey and today I’m interviewing Jeremy Howard and Hamel Husain about nbdev, a library for turning Jupyter notebooks into Python libraries. Interview Introductions How did you get introduced to Python? Can you start by describing what nbdev is and the goals of the project? What is the story behind how and why it got started? Who is the target audience for the nbdev project? How does that focus influence the features and design of nbdev? What do you see as the primary challenges of building and collaborating on projects written in notebooks? What are some of the other projects that are working to simplify or improve the experience of using notebooks? How does nbdev compare to or complement those other tools? Can you describe how nbdev is implemented? How has the design and goals of the project evolved since it was first started? What is the workflow of someone who is using nbdev? At what point in the lifecycle of a notebook oriented project should someone start integrating nbdev? How does nbdev scale when working on a project that spans multiple notebooks/modules? How does working in a notebook environment change your approach to software development and project design? What are the most interesting, innovative, or unexpected ways that you have seen nbdev used? What are the most interesting, unexpected, or challenging lessons that you have learned from working on nbdev? When is nbdev the wrong choice? What do you have planned for the future of the project? Keep In Touch Jeremy LinkedIn @jeremyphoward on Twitter jph00 on GitHub Hamel hamelsmu on GitHub Website @HamelHusain on Twitter LinkedIn Picks Tobias Rivals! Frenemies Who Changed The World Jeremy Chess Hamel Moonwalking With Einstein by Joshua Foer (affiliate link) Closing Announcements Thank you for listening! Don’t forget to check out our other show, the Data Engineering Podcast for the latest on modern data management. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you’ve learned something or tried out a project from the show then tell us about it! Email [email protected]) with your story. To help other people find the show please leave a review on iTunes and tell your friends and co-workers Join the community in the new Zulip chat workspace at pythonpodcast.com/chat Links nbdev fast.ai GitHub Perl Fastmail R Studio R Markdown Literate Programming fastcore JupyterLab nteract Jupyter Voilà GitHub Actions Sphinx Google Colab Working In Public by Nadia Eghbal (affiliate link) Jekyll Hugo Cython Podcast Episode The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA