Python Bytes

Watch the live stream: Watch on YouTube About the show Sponsored by us: Check out the courses over at Talk Python And Brian’s book too! Special guest: Erik Christiansen Michael #1: Fickling via Oli A Python pickling decompiler and static analyzer Pickled ML models are becoming the data exchange and workflow of ML Analyses pickle files for security risks - It can also remove or insert [malicious] code into pickle files... Created by a security firm, it can be a useful defensive or offensive tool. Perhaps it is time to screen all pickles? >>> import ast >>> import pickle >>> from fickling.pickle import Pickled >>> print(ast.dump(Pickled.load(pickle.dumps([1, 2, 3, 4])).ast, indent=4)) Module( body=[ Assign( targets=[ Name(id='result', ctx=Store())], value=List( elts=[ Constant(value=1), Constant(value=2)

Watch the live stream: Watch on YouTube About the show Sponsored by us: Check out the courses over at Talk Python And Brian’s book too! Special guest: Paul Everitt Brian #1: Why I use attrs instead of pydantic Tin Tvrtković, @tintvrtkovic attrs vs dataclasses Since dataclasses are a strict subset of attrs functionality. Recommend using attrs in most cases over dataclasses attrs is faster, has more features, releases more frequently, offers over a wider range of Python versions. attrs vs Pydantic attrs is a library for generating the boring parts of writing classes; Pydantic is that but also a complex validation library. a structuring/unstructuring library, ex converting to json and back attrs has opt-in validation that you have more control over cattrs can be used for structuring/unstructuring converters are opt-in for attrs, built into Pydantic, and can be wrong. example using Pendulum that Pydantic mishandles Summary attrs + cattrs + validators where necessary, converters where necessa

See the full show notes for this episode on the website at pythonbytes.fm/247.

Watch the live stream: Watch on YouTube About the show Sponsored by us: Check out the courses over at Talk Python And Brian’s book too! Special guest: David Smit Brain #1: mktestdocs Vincent D. Warmerdam Tutorial with videos Utilities to check for valid Python code within markdown files and markdown formatted docstrings. Example: import pathlib import pytest from mktestdocs import check_md_file @pytest.mark.parametrize('fpath', pathlib.Path("docs").glob("**/*.md"), ids=str) def test_files_good(fpath): check_md_file(fpath=fpath) This will take any codeblock that starts with ```python and run it, checking for any errors that might happen. Putting assert statements in the code block will actually check things. Other examples in README.md for markdown formatted docstrings from functions and classes. Suggested usage is for code in mkdocs documentation. I’m planning on trying it with blog posts. Michael #2: Redis powered queues (QR3) via Scot Hacker QR queues store

Watch the live stream: Watch on YouTube About the show Sponsored by us: Check out the courses over at Talk Python And Brian’s book too! Special guest: Juan Pedro Araque Espinosa (Youtube Chanel: Commit that Line) Michael #1: State of the community (via Jet Brains) This report presents the combined results of the fifth annual Developer Ecosystem Survey conducted by JetBrains Not just Python, but all of us Python is more popular than Java in terms of overall usage, while Java is more popular than Python as a main language. The 5 fastest growing languages are Python, TypeScript, Kotlin, SQL, and Go. A majority of the respondents (71%) develop for web backend. Does fall into the trap of “Hi, I’m a CSS developer, nice to meet you” though Women are more likely than men to be involved in data analysis, machine learning, and UX/UI design or research. Women are less likely than men to be involved in infrastructure development and DevOps, system administration, or Deployment. Brian #2: Cornell - record &a

Watch the live stream: Watch on YouTube About the show Sponsored by us: Check out the courses over at Talk Python And Brian’s book too! Special guest: Brandon Braner Brain #1: pip Environmental Variables The problem with snakes on a plane → no internet Situation: want to work on some code on a plane project set up to use tox, which creates venvs, and pip installs dependencies from pypi.org. but… no internet Preflight work: run tox with an internet connection copy everything from all of the “site-packages” directories under “.tox” into a “wheels” directory or whatever you want to call it. set environmental variables: PIP_FIND_LINKS=file:/Users/okken/wheels PIP_NO_INDEX=1 Try this out first BEFORE getting on the plane, to make sure you didn’t miss anything. In flight: tox works fine now, using local dir as index Thanks Paul Ganssle for helping with this. All command line flags for pip are available as env variables: “pip’s command line options can be set with environment variables using the f

Watch the live stream: Watch on YouTube About the show Sponsored by us: Check out the courses over at Talk Python And Brian’s book too! Special guest: Simon Willison Michael #1: MongoDB 5 Native Time Series: Designed for IoT and financial analytics, our new time series collections, clustered indexing, and window functions make it easier, faster, and lower cost to build and run time series applications MongoDB automatically optimizes your schema for high storage efficiency, low latency queries, and real-time analytics against temporal data. The Versioned API future-proofs your applications. You can fearlessly upgrade to the latest MongoDB releases without the risk of introducing backward-breaking changes that require application-side rework New MongoDB Shell we have introduced syntax highlighting, intelligent auto-complete, contextual help and useful error messages creating an intuitive, interactive experience for MongoDB users (use mongosh rather than mongo on the CLI). Also launched preview releas

Watch the live stream: Watch on YouTube About the show Sponsored by us: Check out the courses over at Talk Python And Brian’s book too! Special guest: Al Sweigart Brian #1: just From a tweet by Jeff Triplett “just is a handy way to save and run project-specific commands. Commands, called recipes, are stored in a file called justfile with syntax inspired by make Just is a command runner, not a build system, so it avoids much of Make’s complexity and idiosyncrasies. No need for .PHONY recipes! Linux, MacOS, and Windows are supported with no additional dependencies.” It’s written in Rust. My favorite differences: a couple spaces is fine, no need for tabs Recipes can accept command line arguments Recipes can be listed from the command line. Recipes can be written in arbitrary languages, like Python hello: echo "Hello World!" pyhello: #!/usr/bin/env python3 print('Hello from python!') Michael #2: Strong Typing via Roman Right (Beanie) Decorator which checks whether th

Watch the live stream: Watch on YouTube About the show Sponsored by us: Check out the courses over at Talk Python And Brian’s book too! Special guest: Jay Miller Michael #1: Autosync all branches of a fork Use GitHub actions to keep your fork in sync Step 1: make changes in a separate branch (a branch other than main) to keep the working tree clean and avoiding conflicts with upstream Step 2: Add a new workflow under the “actions” section. We are going to follow the Fork-Sync-With-Upstream-action from the Actions Marketplace. Copy the YAML in the article being careful to use the right repo/branch names Step 3: click on Start Commit and Commit new file and that's it! See your running workflow in the actions tab Brain #2: Measuring memory usage in Python: it’s tricky! Itamar Turner-Trauring Nice, easy to follow discussion of memory Cool example to allocate 3 GB arr = np.ones((1024, 1024, 1024, 3), dtype=np.uint8) that’s a 4 dimensional array of bytes, 1k x 1k x 1k x 3 “Resident Memory” measure

See the full show notes for this episode on the website at pythonbytes.fm/240.

Watch the live stream: Watch on YouTube About the show Sponsored by us: Check out the courses over at Talk Python And Brian’s book too! Special guest: Nick Muoh Brain #1: ormar : an async mini ORM for Python, with support for Postgres, MySQL, and SQLite. suggested by John Hagen From John: “It's a really cool ORM that combines Pydantic models and SQL models into a single definition. What is great about this, is it can be used to reduce repetitive duplication between Models for an ORM and the Pydantic Models that FastAPI needs to describe serialization. … If you have very pure-data heavy abstractions where your input and outputs through the API are roughly equivalent to your database, this helps you avoid needing to duplicate tons of SQLAlchemy classes and Pydantic that look identical and now you need to keep them in sync (DRY issue).” Michael #2: No module named via Garett Dunn Website: nomodulenamed.com Get an error like Python Error: No module named dateutil, maybe you need pip install python_

Watch the live stream: Watch on YouTube About the show Sponsored by Sentry: Sign up at pythonbytes.fm/sentry And please, when signing up, click Got a promo code? Redeem and enter PYTHONBYTES Special guest: Julia Signell Brain #1: Practical SQL for Data Analysis Haki Benita Pandas is awesome, but … “In this article I demonstrate how to use SQL to perform fast and efficient data analysis.” First part of the article. SQL is faster than Pandas But they are great together Then tons of examples showing exactly how to best use SQL queries and Pandas in data analysis:: Basics including random data and sampling Descriptive statistics Subtotals including rollup and groupign sets Pivot tables, both conditional expressions and aggregate expressions Running and cumulative agregation Linear Regression Interpolation Super cheat sheet for useful SQL queries Michael #2: Git Blame in your Python Tracebacks via Ruslan Portnoy, by Ofer Koren Helpful Modules: traceback & linecache traceback uses linecache,

Watch the live stream: Watch on YouTube About the show Sponsored by Sentry: Sign up at pythonbytes.fm/sentry And please, when signing up, click Got a promo code? Redeem and enter PYTHONBYTES Special guest: Mike Groves Michael #1: Textual Textual (Rich.tui) is a TUI (Text User Interface) framework for Python using Rich as a renderer. Rich TUI will integrate tightly with its parent project, Rich. This project is currently a work in progress and may not be usable for a while. Brian #2: Pinning application dependencies with pip-tools compile via John Hagen pip-tools has more functionality than this, but compile alone is quite useful Start with a loose list of dependencies in requirements.in: rich Can have things like >= and such if you have fixed dependencies. Now pip install pip-tools, and pip-compile requirements.in or python -m piptools compile requirements.in both have same effect. Now you’ll have a requirements.txt file with pinned dependencies: # autogenerated by: pip-comp

Watch the live stream: Watch on YouTube About the show Sponsored by Sentry: Sign up at pythonbytes.fm/sentry And please, when signing up, click Got a promo code? Redeem and enter PYTHONBYTES Special guest: Anastasiia Tymoshchuk Brian #1: Using accessible colors, monolens & CMasher Tweet by Matthew Feickert, @HEPfeickert “I need to give some serious praise to fellow Scikit-HEP dev Hans Dembinski on his excellent monolens tool for interactive simulation of kinds of color blindness. It works really quite well and the fact that is a pipx install away is awesome! monolens lets you “view part of your screen in greyscale or simulated colorblindness” So simple. Just pops up a box that you can drag around your monitor and view stuff in greyscale. Reply tweet by Niko, @NikoSercevic “I mean to use cmasher so I know it’s cb friendly” CMasher : “Scientific colormaps for making accessible, informative and cmashing plots” Provides a collection of scientific colormaps and utility functions to be used b

Watch the live stream: Watch on YouTube About the show Sponsored by Sentry: Sign up at pythonbytes.fm/sentry And please, when signing up, click Got a promo code? Redeem and enter PYTHONBYTES Special guest: Vincent D. Warmerdam koaning.io, Research Advocate @ Rasa and maintainer of a whole bunch of projects. Intro: Hello and Welcome to Python Bytes Where we deliver Python news and headlines directly to your earbuds. This is episode 235, recorded May 26 2021 I’m Brian Okken [HTML_REMOVED] [HTML_REMOVED] Brian #1: Flask 2.0 articles and reactions Change list Async in Flask 2.0 Patrick Kennedy on testdriven.io blog Great description discussion of how the async works in Flask 2.0 examples how to test async routes An opinionated review of the most interesting aspects of Flask 2.0 Miguel Grinberg video covers route decorators for common methods, ex @app.post(``"``/``"``) instead of @app.route("/", methods=["POST"]) web socket support async support Also includes some extensions Miguel has written

Watch the live stream: Watch on YouTube About the show Sponsored by Sentry: Sign up at pythonbytes.fm/sentry And please, when signing up, click Got a promo code? Redeem and enter PYTHONBYTES Special guest: Dr. Becky Smethurst Brian #1: Powering the Python Package Index in 2021 Dustin Ingram A lot has changed in 5 years since the previous write-up From 3 people to 3 maintainers/admins 5 moderators 3 commiters Companies donate about $1.8M per month in services Fastly, mostly Google Cloud ~ $10K AWS ~ $7K Also Statuspage, Sentry, Datadog, Digicert, Pingdom Awesome grants to fund projects rewrite of PyPI Localization, internationalization, API tokens and 2FA Malware Detection and Update Framework Foundational Tool Improvements & Productionized Malware Detection Support Staff (a project manager) Growth, now up to (per day) 1.7 B requests pypi 55.4 TB pypi Next steps FUNDABLES.md, which is a non-exhaustive wishlist of large projects we’d like to see happen become a member, donate, or volunt

Watch the live stream: Watch on YouTube About the show Sponsored by us! Support our work through: Our courses at Talk Python Training pytest book Patreon Supporters Special guest: Marlene Mhangami Brian #1: readme.so Recommended by Johnny Metz This is not only useful, it’s fun Interactively create a README.md file Suggested sections great There are lots of sections though, so really only pick the ones you are willing to fill in. I think this is nicer than the old stand by of “copying the README.md of another project” because that other project might not have some of these great sections, like: Acknowledgements API Reference Authors FAQ Features Logo Roadmap Usage/Examples Running Tests Note, these sections are listed in alphabetical order, not necessarily the right order for how they should go in your README.md Produces a markdown file you can copy or download Also an editor so you can edit right there. (But I’d probably throw together the skeleton with dummy text and edit it in something with v

Watch the live stream: Watch on YouTube About the show Sponsored by us! Support our work through: Our courses at Talk Python Training pytest book Patreon Supporters Special guest: Annette Lewis Brian #1: Sphinx Themes Gallery update Curated and maintained by @pradyunsg and @shirou. I actually don’t know what it looked like before, but this is great. I’m working on my first real Sphinx project, so this is awesome to have. Features: Main image for each theme shows what theme looks like in wide, narrow, and phone layout Demos (click on an image): Main page that shows you quick start: install and config theme name Link to theme documentation Example of Navigation Kitchen sink paragraph level markup including inline, math, meta, blocks, code with sidebars, references, directives, footnotes, and more API documentation example essential if you are using this for documenting code Lists and tables Michael #2: Mongita - Like SQLite but for MongoDB Mongita is a lightweight embedded document data

Watch the live stream: Watch on YouTube About the show Sponsored by us! Support our work through: Our courses at Talk Python Training pytest book Patreon Supporters Special guests: Cecil Phillip Brian #1: For-Else: A Weird but Useful Feature in Python Yang Zhou After a for loop, you can put an else block. The else block only executes when there is no break in the loop. If the loop got all the way to the end, and off the end, the else block will run. First, I’m not used to putting break or else anywhere in my Python code, so I’m also curious why you’d want to do this. Yang explains the feature, then talks about 3 scenarios for use: Iterate and find items without needing a flag variable. break when you find what you are looking for, and the else only runs if you didn’t find it. Help to break out of nested loops I’m still confused by this one Help to handle exceptions Kind of a cool use. try/except in a for loop. Have a break in the except block. Then the else block will be fore code where y

Watch the live stream: Watch on YouTube About the show Sponsored by us! Support our work through: Our courses at Talk Python Training pytest book Patreon Supporters Special guests: Peter Kazarinoff Brian #1: calmcode.io by Vincent D. Warmerdam Suggested by Rens Dimmendaal Great short intro tutorials & videos. Not deep dives, but not too shallow either. Suggestions: pytest rich datasette I watched the whole series on datasette this morning and learned how to turn a csv data file into a sqlite database use datasette to open a server to explore the data filter the data visualize the data with datasette-vega plugin and charting options learn how I can run random SQL, but it’s safe because it’s read only use it as an API that serves either CSV or json deploy it to a cloud provider by wrapping it in a docker container and deploying that add user authentication to protect the service explore tons of available data sets that have been turned into live services with datasette Michael #2: Natural