Python Bytes

Watch the live stream: Watch on YouTube About the show Sponsored by us! Support our work through: Our courses at Talk Python Training pytest book Patreon Supporters Special guests: Gwendolyn Faraday Gwendolyn’s YouTube Brian #1: Coverage.py (5.6b1) and third-party code Problems If you put your virtual environment in the same directory as your code, and try to run coverage, it’s tricky to get coverage to not attempt to cover everything in your venv also. Or even just running coverage run -m pytest with no --source specified, it just kinda reports on everything, even stuff in site-packages, not just your code. Solution pip install coverage==5.6b1 As of 5.6b1, coverage knows where third party code is and doesn’t measure it. Super awesome Also, it’s still beta. Net wants help testing it out and making sure it works right. I’m curious if it still works right with pytest plugins and such, so I’ll be testing a bunch of stuff to make sure it still makes sense. Michael #2: So you want your own P

Watch the live stream: Watch on YouTube About the show Sponsored by us! Support our work through: Our courses at Talk Python Training pytest book Patreon Supporters Special guest Guy Royse Brian #1: How to make an awesome Python package in 2021 Anton Zhiyanov, @ohmypy Also thanks John Mitchell, @JohnTellsAll for posting about it. Great writing taking you through everything in a sane order. Stubbing a project with just .gitignore and a directory with a stub __init__.py. Test packaging and publishing use flit init to create initial pyproject.toml set up your ~/.pypirc file publish to the test repo Make the real thing make an implementation publish Extras Adding README.md & CHANGELOG.md and updating pyproject.toml to include README.md and a Python version selector. Adding linting and testing with pytest, tox, coverage, and others Building in the cloud with GH Actions, Codecov, Code Climate Adding badges Task automation with a Makefile Publishing to PyPI from a GH Action Missing (but possibl

Watch the live stream: Watch on YouTube About the show Sponsored by us! Special guest: Micaela Reyes Brian #1: Number One, that's "retract plank," not "remove plank." Yanking vs removing versions on PyPI https://twitter.com/nedbat/status/1376901333958201352?s=20 https://pypi.org/help/#yanked see also https://doughellmann.com/posts/so-youve-released-a-broken-package-to-pypi-what-do-you-do-now/ Michael #2: SQLAlchemy 1.4.0 Released Exciting: 1st release to properly support an async API Has a new select() + execute() rather than session.query() API Intended to unify Core and ORM. See new vs. old API compared. Requires aiosqlite for async API + SQLite: conn_str = 'sqlite+aiosqlite:///filename' Micaela #3: django-tenants by Tom Turner Multi-tenancy Implementation for Django (typically for SaaS websites e.g. Shopify) currently on v3.2.1 (Aug 2020) release Requirements: Django 2 and PostgreSQL It was largely based on django-tenant-schemas library Data Architecture: shared database, separate schema f

Special guests: Kelly Schuster-Paredes Sean Tibor Watch on YouTube Brian #1: DataClass vs NamedTuple vs Object: A Battle of Performance in Python Jack Song I’ve been using dataclass for a while now and love it. For some reason, I thought I heard there was some performance hit from them, so I was a bit worried before reading this. Jack came up with “a performance tests to compare the different size and speed when creating, reading and executing functions for Object, NamedTuple and the new DataClass introduced in Python 3.7” Object NamedTuple DataClass create 2.94 µs 2.01 µs 2.34 µs read property 24.7 ns 26.9 ns 24.7 ns nested property 48.1 ns 75.8 ns 52.1 ns execute function 829 ns 946 ns 821 ns size 56 bytes 80 bytes 56 bytes Marvelous. Dataclass is still awesome. At the very least, it’s on the same order of size and speed as other structures. Further questions: This was a limited bit of code, and performance metrics always depen

Sponsored by Linode! pythonbytes.fm/linode Special guest: Sebastian Witowski Watch on YouTube Brian #1: Raspberry Pi Pico Release Announcement A review $4 microcontroller Small Extremely low power needs. Built on RP2040, a brand-new chip developed by Raspberry Pi Related: Mu : codewith.mu, 1.1.0-beta.2 Mu is “a simple Python editor for beginner programmers.” 1.1.0 support new boards, including Pico, Lego Spike, plus lots of new fixes. Michael #2: New MongoDB ODM: Beanie via PyCoders Beanie - is an asynchronous ODM for MongoDB, based on Motor and Pydantic. Very new but also very exciting. Main component of Beanie is Pydantic. It helps to implement the main feature - data structuring. Beanie Document - is an abstraction over the Pydantic BaseModel that allows working with Python objects at the application level and JSON objects at the database level. Example, classes: class TagColors(str, Enum): RED = "RED" BLUE = "BLUE" GREEN = "GREEN" class Tag(BaseModel):

Special guest: Calvin Hendryx-Parker Live stream Watch on YouTube Michael #1: AWSimple by James Abel AWSimple is a more object oriented interface on top of boto3 for some of the common “serverless” AWS services: S3, DynamoDB, SNS, and SQS. Features: Simple Object Oriented API on top of boto3 One-line S3 file write, read, and delete Automatic S3 retries Locally cached S3 accesses True file hashing (SHA512) for S3 files (S3's etag is not a true file hash) DynamoDB full table scans (with local cache option) DynamoDB secondary indexes Built-in pagination (e.g. for DynamoDB table scans and queries). Always get everything you asked for. Can automatically set SQS timeouts based on runtime data (can also be user-specified) Caching: S3 objects and DynamoDB tables can be cached locally to reduce network traffic, minimize AWS costs, and potentially offer a speedup. Brian #2: coverage and installed packages I’ve covered coverage.py a lot on Test & Code, starting with episode 12, and even talked about it

Sponsored by Datadog: pythonbytes.fm/datadog Special guest: Sebastián Ramírez Live stream Watch on YouTube Brian #1: Python Developers Survey 2020 Results Using Python for? Lots of reductions in percentages. Increases in Education, Desktop, Games, Mobile, and Other Python 3 vs 2 94% Python3 vs 90% last year Python 3.8 has 44% of Python 3 usage, 3.5 or lower down to 3% environment isolation 54% virtualenv (I assume that includes venv) 32% Docker 22% Conda Web frameworks 46% Flask 43% Django 12% FastAPI … 2% Pyramid :( … Unit testing 49% pytest 28% unittest 13% mock OS 68% Linux, 48% Windows, 29% Mac, 2% BSD, 1% other CI: Gitlab, Jenkins, Travis, CircleCI … (Where’s GH Actions?) Editors: PyCharm, VS Code, Vim, … Lots of other great stuff in there Michael #2: Django Ninja - Fast Django REST Framework via Marcus Sharp and Adam Parkin (Codependent Codr) independently Django Ninja is a web framework for building APIs with Django and Python 3.6+ type hints. This project was heavily inspire

Sponsored by Linode! pythonbytes.fm/linode Special guest: Greg Herrera YouTube live stream for viewers: Watch on YouTube Michael #1: boto type annotations via Michael Lerner boto3's services are created at runtime IDEs aren't able to index its code in order to provide code completion or infer the type of these services or of the objects created by them. Type systems cannot verify them Even if it was able to do so, clients and service resources are created using a service agnostic factory method and are only identified by a string argument of that method. boto3_type_annotations defines stand in classes for the clients, service resources, paginators, and waiters provided by boto3's services. Example with “bare” boto3: Example with annotated boto3: Brian #2: How to have your code reviewer appreciate you By Michael Lynch Suggested by Miłosz Bednarzak Actual title “How to Make Your Code Reviewer Fall in Love with You” but

Sponsored by Datadog: pythonbytes.fm/datadog Special guest: Brett Cannon Brian #1: Keeping up with Rich Will McGugan has been building Rich It looks like it’s on its way to becoming a full fledged TUI (text user interface) December: Live view: no blog post on that, I don’t think. January: Tree view: Rendering a tree view in the terminal with Python and Rich February: Layouts: Building Rich terminal dashboards fun fullscreen.py example, uses Live view Also, python -m rich will display a demo screen that shows tons of the stuff that Rich can do Many of the features also have a stand alone demo built in, like: $ python -m rich.layout $ python -m rich.tree $ python -m rich.live Although I haven’t figured out how to kill the live demo. it doesn’t seem to time out, and it eats Ctrl-C in my terminal. I’d really like to use Rich for interactive stuff, like keyboard interrupts and arrow keys and tab and such. It’d be fun. Which brings me to the bottom right corner of the python -m rich output.

Sponsored by Datadog: pythonbytes.fm/datadog Special guest: Hannah Stepanek Watch on YouTube Michael #1: We Downloaded 10,000,000 Jupyter Notebooks From Github – This Is What We Learned by Alena Guzharina from JetBrains Used the hundreds of thousands of publicly accessible repos on GitHub to learn more about the current state of data science. I think it’s inspired by work showcased here on Talk Python. 2 years ago there were 1,230,000 Jupyter Notebooks published on GitHub. By October 2020 this number had grown 8 times, and we were able to download 9,720,000 notebooks. 8x growth. Despite the rapid growth in popularity of R and Julia in recent years, Python still remains the most commonly used language for writing code in Jupyter Notebooks by an enormous margin. Python 2 went from 53% → 11% in the last two years. Interesting graphs about package usage Not all notebooks are story telling with code: 50% of notebooks contain fewer than 4 Markdown cells and more than 66 code cells. Although there are some out

Sponsored by us! Support our work through: Our courses at Talk Python Training pytest book Patreon Supporters Special guest: Jennifer Stark - @_JAStark & guest on talkpython.fm/259 Watch on YouTube Brian #1: Do you really need a virtualenv? Frost Ming doesn’t think so, based on the article You don't really need a virtualenv The link slug is “introducing-pdm”, which I think would be a better title, but the first did work to get people to talk about it. Also, “Try PEP 582 today” may have been appropriate. Teaching new people is a problem: Telling them to first type python -m venv venv Then type source venv/bin/activate or . venv/bin/activate Unless you’re on windows, then type venv\scripts\activate.bat Then type pip install -r requirements.txt Yeah. It’s not pretty, not fun, and good luck not having anyone ask questions about why this is necessary. Also the Python version is specified in the venv. So if you upgrade Python versions, what happens to existing venvs? The article also discusses level

Sponsored by Datadog: pythonbytes.fm/datadog Special guest: Jeremy Tanner Watch on YouTube Brian #1: Constant Folding in Python Arpit Bhayani Constant Folding is when a language replaces constant expressions at compile time rather than computing them at runtime. CPython does this while creating the bytecode. We can use dis to see it in action >>> import dis >>> dis.dis("day_sec = 24 * 60 * 60") 1 0 LOAD_CONST 0 (86400) 2 STORE_NAME 0 (day_sec) 4 LOAD_CONST 1 (None) 6 RETURN_VALUE Python tries to fold lots of constants, but not all. Seems to be based on size >>> x = 2 ** 64 # folded >>> x = 4 ** 64 # not folded >>> a = "-" * 4096 # folded >>> a = "-" * 4097 # not folded Discussion continues with a discussion of how CPython folding is implemented recursively and elegantly. Key takeaway for me: Remember to n

Sponsored by Linode! pythonbytes.fm/linode Special guest: Ogi Moore Watch the live stream on YouTube. Michael #1: diskcache via Ian Maurer Python disk-backed cache (Django-compatible). Faster than Redis and Memcached. Pure-Python. The cloud-based computing of 2020 puts a premium on memory. Gigabytes of empty space is left on disks as processes vie for memory. Among these processes is Memcached (and sometimes Redis) which is used as a cache. Wouldn't it be nice to leverage empty disk space for caching? Features: Pure-Python Fully Documented Benchmark comparisons (alternatives, Django cache backends) 100% test coverage Hours of stress testing Performance matters Django compatible API Thread-safe and process-safe Supports multiple eviction policies (LRU and LFU included) Keys support "tag" metadata and eviction Developed on Python 3.8 Tested on CPython 3.5, 3.6, 3.7, 3.8 Tested on Linux, Mac OS X, and Windows Tested using Travis CI and AppVeyor CI Brian #2: TOML is 1.0.0 now. What does that mean for

Sponsored by Datadog: pythonbytes.fm/datadog Special guest: Jousef Murad, Engineered Mind podcast (audio, video) Watch on YouTube Brian #1: pip search. Just don’t. pip search [query] is supposed to “Search for PyPI packages whose name or summary contains [query]” The search feature looks like it’s going to be removed and the PyPI api for it removed. Alternative, and better approach, just manually look at pypi.org and search for stuff. Right now it does this: $ pip search pytest ERROR: Exception: Traceback (most recent call last): ... [longish traceback ommited] --- xmlrpc.client.Fault: [Fault -32500: "RuntimeError: PyPI's XMLRPC API has been temporarily disabled due to unmanageable load and will be deprecated in the near future. See https://status.python.org/ for more information."] The Python Infrastructure status page says, as of Jan 12: “Update - The XMLRPC Search endpoint remains disabled due to ongoing request volume. As of this update, there has been no reduction in inbou

Sponsored by us! Support our work through: Our courses at Talk Python Training pytest book Patreon Supporters Special guest: Jason McDonald Watch on YouTube Michael #1: 5 ways I use code as an astrophysicist Video by Dr. Becky (i.e. Dr Becky Smethurst, an astrophysicist at the University of Oxford) She has a great YouTube channel to check out. #1: Image Processing (of galaxies from telescopes) Noise removal #2: Data analysis Image features (brightness, etc) One example: 600k “rows” of galaxy properties #3: Model fitting e.g. linear fit (visually as well through jupyter) e.g. Galaxies and their black holes grow in mass together Color of galaxies & relative star formation #4: Data visualization #5: Simulations Beautiful example of galaxies colliding Star meets black hole Brian #2: A Visual Intro to NumPy and Data Representation Jay Alammar I’ve started using numpy more frequently in my own work. Problem: I think of np.array like a Python list. But that’s not right. This visualization gu

2020 will be one for the history books, won't it? I've put together a great group to look back on 2020 - from the Python perspective. Join Brian and Michael along with Cecil Phillip, Ines Montani, Jay Miller, Paul Everitt, Reuven Lerner, and Matt Harrison for a light-hearted and fun look back on the major Python events of 2020. Video version of this episode: Watch on YouTube Guests Cecil Phillip Ines Montani Jay Miller Paul Everitt Reuven Lerner Matt Harrison

Sponsored by us! Support our work through: Our courses at Talk Python Training Test & Code Podcast Patreon Supporters Special guest: Anthony Shaw Stream on YouTube as well Watch on YouTube Michael #1: Django Ledger Project by Miguel Sanda The mission is to provide free and open source accounting software that could easily replace commercial alternatives like QuickBooks. Django Ledger supports: Chart of Accounts. Financial Statements (Income Statement & Balance Sheets). Automatic financial ratio & insight calculations. Multi tenancy. Hierarchical entity management. Self-contained Ledgers, Journal Entries & Transactions. Financial Activities Support (operational/financial/investing). Basic OFX & QFX file import. Bills & Invoices with optional progressible functionality. Basic navigational templates. Entity administration & entity manager support. Bank Accounts. Not quite ready for production. This project is actively looking for contributors. Any financial and/or accounti

Sponsored by us! Support our work through: Our courses at Talk Python Training Test & Code Podcast Patreon Supporters Special guest: Shari Eskenas Catch the video edition live stream on YouTube Watch on YouTube Be part of the episode by subscribing and “smashing that bell” over at pythonbytes.fm/youtube Brian #1: pytest 6.2 is out pytester fixture for plugin testing. Like testdir, but a better name, and uses pathlib.Path objects. verbose mode now shows the reason a test was skipped in the terminal line after the SKIPPED, XFAIL, or XPASS Can use monkeypatch as a context manager with .context() and it’s available both in test and fixture functions, but also in helper functions by using pytest.Monkeypatch.context(). import os from contextlib import contextmanager import pytest def test_foo_1(monkeypatch): with monkeypatch.context() as mp: mp.setenv("foo", "bar") assert os.getenv("foo") == "bar" @contextmanager def some_func(): w

Sponsored by Techmeme Ride Home podcast: pythonbytes.fm/ride Special guest: Matthew Feickert Live streamed on Youtube: youtube.com/watch?v=ug3OR-BQ7Hw Brian #1: Introducing FARM Stack - FastAPI, React, and MongoDB Aaron Basset Describes a todo CRUD application, available on github, to show the concepts in action. Animated gif showing how to use the FastAPI interactive documentation to understand the API in a browser. Shows How app connects to routes and endpoints. The run call with uvicorn to get an async even loop going Connecting to a MongoDB database Defining models and how easy it is to set up a schema. Discusses routes and endpoints and how to hook up all the CRUD endpoints. The React bit is an application that changes as you modify the elements through the endpoint interactive session. Very cool way to work and get something running fast. Michael #2: py-applescript How would you like to automate your mac with Python rather than the dreadful AppleScript? py-applescript is an easy-to-use Pytho

The live stream recording on YouTube. Special guest: Jay Miller Sponsored by us! Support our work through: Our courses at Talk Python Training Test & Code Podcast Patreon Supporters Brian #1: Analyzing Kickstarter Campaigns with Python Data Science Tools Article title: “Kickstarter Projects — Do They Succeed?” Aditya Patkar Using a Kaggle dataset of 378,661-ish projects up to 2018. Looks at using pandas data frames to explore the data. Using .describe() data frame method to learn a lot. Uses matplotlib and seaborn to analyze the data further. Odd statement that I’m not sure is straight faced or a really dry joke: “The data from 1970 seems to be bad or insignificant data.” Examples of using heat maps, line graphs, bar charts, to look at different aspects. Some results: 35.64% of projects are successful (meaning goal hit) tech asks for the most for goals, and has the highest average per backer. Comics has the lowest pledged amount per backer average. Nice that you can use the techniques to ask yo