Python Bytes

Watch the live stream: Watch on YouTube About the show Sponsored by Datadog: pythonbytes.fm/datadog Special guest: Brian Skinn (Twitter | Github) Michael #1: OpenBB wants to be an open source challenger to Bloomberg Terminal OpenBB Terminal provides a modern Python-based integrated environment for investment research, that allows an average joe retail trader to leverage state-of-the-art Data Science and Machine Learning technologies. As a modern Python-based environment, OpenBBTerminal opens access to numerous Python data libraries in Data Science (Pandas, Numpy, Scipy, Jupyter) Machine Learning (Pytorch, Tensorflow, Sklearn, Flair) Data Acquisition (Beautiful Soup, and numerous third-party APIs) They have a discord community too BTW, seem to be a successful open source project: OpenBB Raises $8.5M in Seed Round Funding Following Open Source Project Gamestonk Terminal's Success Great graphics / gallery here. Way more affordable than the $1,900/mo/user for the Bloomberg Terminal Brian #2: Python f

Watch the live stream: Watch on YouTube About the show Sponsored by: Microsoft for Startups Founders Hub. Special guest: Vuyisile Ndlovu Brian #1: dunk - a prettier git diff Darren Burns Uses Rich “⚠️ This project is very early stages” - whatever, I like it. Recommendation is to use less as a pager for it git diff | dunk | less -R Michael #2: Is your Python code vulnerable to log injection? via Adam Parkin Let’s just appreciate log4jmemes.com for a moment Ok, now we can talk about Python We can freak our the logging with line injection "hello'.\nINFO:__main__:user 'alice' commented: 'I like pineapple pizza" Results in two lines for one statement INFO:__main__:user 'bob' commented: 'hello'. INFO:__main__:user 'alice' commented: 'I like pineapple pizza'. The safest solution is to simply not log untrusted text. If you need to store it for an audit trail, use a database. Alternatively, structured logging can prevent newline-based attacks. Padding a ton? One such case is abusing

Watch the live stream: Watch on YouTube About the show Sponsored by: Microsoft for Startups Founders Hub. Special guest: Thomas Gaigher, creator/maintainer pypyr taskrunner Michael #1: March Package Madness via Chris May Start with 16 packages They battle it out 2-on-2 in elimination rounds Voting is once a week So go vote! Brian #2: nbpreview “A terminal viewer for Jupyter notebooks. It’s like cat for ipynb files.” Some cool features pretty colors by default piping strips formatting, so you can pass it to grep or other post processing automatic paging syntax highlighting line numbers and wrapping work nicely markdown rendering images converted to block, character, or dots (braille) dataframe rendering clickable links Thomas #3: pyfakefs A fake file system! It intercepts all calls that involve the filesystem in Python - e.g open(), shutil, or pathlib.Path. This is completely transparent - your functional code does not know or need to know that under the hood it's been disconnected from

Watch the live stream: Watch on YouTube About the show Sponsored by FusionAuth: pythonbytes.fm/fusionauth Special guest: Ian Hellen Brian #1: gensim.parsing.preprocessing Problem I’m working on Turn a blog title into a possible url example: “Twisted and Testing Event Driven / Asynchronous Applications - Glyph” would like, perhaps: “twisted-testing-event-driven-asynchrounous-applications” Sub-problem: remove stop words ← this is the hard part I started with an article called Removing Stop Words from Strings in Python It covered how to do this with NLTK, Gensim, and SpaCy I was most successful with remove_stopwords() from Gensim from gensim.parsing.preprocessing import remove_stopwords It’s part of a gensim.parsing.preprocessing package I wonder what’s all in there? a treasure trove gensim.parsing.preprocessing.preprocess_string is one this function applies filters to a string, with the defaults almost being just what I want: strip_tags() strip_punctuation() strip_multiple_whitespaces() str

Watch the live stream: Watch on YouTube About the show Sponsored by Microsoft for Startups Founders Hub. Special guest: Emily Morehouse-Valcarcel Michael #1: Async and await with subprocesses by Fredrik Averpil People know I do all sorts of stuff with async Lots of cool async things are not necessarily built into Python, but our instead third-party packages E.g. files via aiofiles But asyncio has asyncio.create_subprocess_exec Fredrik’s article has some nice examples I started using this for mp3 uploads and behind the scenes processing for us Brian #2: Typesplainer Arian Mollik Wasi, @wasi_master Suggested by Will McGugan Now released a vscode extension for that! Available on vscode as typesplainer Emily #3: Ibis Project via Marlene Mhangami “Productivity-centric Python data analysis framework for SQL engines and Hadoop” focused on: Type safety Expressiveness Composability Familiarity Marlene wrote an excellent blog post as an introduction Works with tons of different backends, either dir

Watch the live stream: Watch on YouTube About the show Sponsored by Microsoft for Startups Founders Hub. Special guest: Anne Barela Brian #1: The Adam Test : 12 Questions for New Dependencies Found through a discussion with Ryan Cheley, who will be on an upcoming episode of Test & Code, talking about Managing Software Teams. The Joel Test dates back to 2000, and some of it is a bit dated. I should probably do a Test & Code episode or pythontest article on my opinions of this at some point. Nice shameless plugs, don’t you think? The Joel Test is 12 questions and is a “highly irresponsible, sloppy test to rate the quality of a software team.” “The Adam Test” is 12 questions “to decide whether a new package we’re considering depending on is well-maintained.” He’s calling it “The Well-Maintained Test”, but I like “The Adam Test” Here’s the test: Is it described as “production ready”? Is there sufficient documentation? Is there a changelog? Is someone responding to bug reports? Are there suf

Watch the live stream: Watch on YouTube About the show Sponsored by Datadog: pythonbytes.fm/datadog Michael #1: Physics Breakthrough as AI Successfully Controls Plasma in Nuclear Fusion Experiment Interesting break through using AI Is Python at the center of it? With enough digging, the anwswer is yes, and we love it! Brian #2: PEP 680 -- tomllib: Support for Parsing TOML in the Standard Library Accepted for Python 3.11 This PEP proposes basing the standard library support for reading TOML on the third-party library tomli Michael #3: Thread local threading.local: A class that represents thread-local data. Thread-local data are data whose values are thread specific. Just create an instance of local (or a subclass) and store attributes on it You can even subclass it. Brian #4: What is a generator function? Trey Hunner Super handy, and way easier than you think, if you’ve never written your own. Really, it’s just a function that uses yield instead of return and supplies one element at a time

Watch the live stream: Watch on YouTube About the show Sponsor: Brought to you by FusionAuth - check them out at pythonbytes.fm/fusionauth Special guest: Calvin Hendryx-Parker Brian #1: Why your mock still doesn’t work Ned Batchelder Some back story: Why your mock doesn’t work a quick tour of Python name assignment The short version of Python Names and Values talk importing difference between from foo import bar and import foo w.r.t mocking punchline: “Mock it where it’s used” Now, Why your mock still doesn’t work talks about using @patch decorator (also applies to @patch.object decorator) and utilizing mock_thing1, mock_thing2 parameters to test you can change the return value or an attribute or whatever. normal mock stuff. But…. the punchline…. be careful about the order of patches. It needs to be @patch("foo.thing2") @patch("foo.thing1") def test_(mock_thing1, mock_thing2): ... Further reading: https://docs.python.org/3/library/unittest.mock.html#patch https://docs.pyt

Watch the live stream: Watch on YouTube About the show Sponsored by us: Check out the courses over at Talk Python And Brian’s book too! Special guest: Steve Dower Michael #1: fastapi-events Asynchronous event dispatching/handling library for FastAPI and Starlette Features: straightforward API to emit events anywhere in your code events are handled after responses are returned (doesn't affect response time) support event piping to remote queues powerful built-in handlers to handle events locally and remotely coroutine functions (async def) are the first-class citizen write your handlers, never be limited to just what fastapi_events provides Brian #2: Ways I Use Testing as a Data Scientist Peter Baumgartner “In my work, writing tests serves three purposes: making sure things work, documenting my understanding, preventing future errors.” Test The results of some analysis process (using assert) Code that operates on data (using hypothesis) Aspects of the data (using pandera or Great Expectations

Watch the live stream: Watch on YouTube About the show Sponsored by Datadog: pythonbytes.fm/datadog Special guest: Dean Langsam Brian #1: A Better Pygame Mainloop Glyph Doing some game programming is a great way to work on coding for early devs (and experienced devs). pygame is a popular package for writing games in Python But… the normal example of a main loop, which listens for events and dispatches actions based on events, has some problems: it’s got a while 1: that wastes power, too much busy waiting looks bad, due to “screen tearing” which is writing to a screen while your in the middle of drawing it This post discusses the problems, and walks through to an async main loop that creates a better gaming experience. Michael #2: awesome sqlalchemy A few notable ones SQLAlchemy-Continuum: Versioning and auditing extension for SQLAlchemy. SQLAlchemy-Utc: SQLAlchemy type to store aware datetime.datetime values. SQLAlchemy-Utils: Various utility functions, new data types and helpers for SQLAlchem

Watch the live stream: Watch on YouTube About the show Sponsored by Datadog: pythonbytes.fm/datadog Special guest: Luciana Brian #1:rich-cli suggested by Lance Reinsmith rich on the command line. why? syntax highlighting rich example.py rich -m README.md use -m for markdown why Will? .md seems clear enough to me. comes with themes. ex: --theme monokai formats json, --json or -j and a bunch of other features I probably won’t use, but you might. alignment, maybe width, yeah, I’ll probably use -w a bunch more In my .zshrc: alias cat='rich --theme monokai' after pipx install rich-cli feel free to tell me that I shouldn’t used cat for looking at file contents. (although, why not?) I’m not, I’m using rich. :) Luciana #2: debugpy - a debugger for Python The debugger we use in the Python extension for VS Code Super heplful features that can save up a lot of time and a lot of folks don’t seem to know about: Conditional breakpoints Helpful when you want the code to break only on a specific co

Watch the live stream: Watch on YouTube About the show Sponsored by us: Check out the courses over at Talk Python And Brian’s book too! Brian #1: (draft) PEP 679 -- Allow parentheses in assert statements Pablo Galindo Salgado This is in draft, not approved, and not scheduled for any release But it seems like a really good idea to me. assert(1 == 2, "seems like it should fail") will always pass currently since the tuple (False,"seems like it should fail") is a non-empty tuple. Current Python will emit a warning >>> assert(1 == 2, "seems like it should fail") [stdin]:1: SyntaxWarning: assertion is always true, perhaps remove parentheses? But really, why not just change the language to allow assert with or without parens. Also would allow multi-line assert statements more easily: assert ( very very long expression, "very very long " "message", ) I hope this is a slam dunk and gets in ASAP. Michael #2: Everything I googled as a dev

Watch the live stream: Watch on YouTube About the show Sponsored by us: Check out the courses over at Talk Python And Brian’s book too! Michael #1: Box: Python dictionaries with advanced dot notation access Want to treat dictionaries like classes? Box. small_box = Box({'data': 2, 'count': 5}) small_box.data == \ small_box['data'] == \ getattr(small_box, 'data') == \ small_box.get('data') There are over a half dozen ways to customize your Box and make it work for you: Check out the new Box github wiki for more details and examples! Superset of dict See Types of Boxes as well Brian #2: Reading tracebacks in Python Trey Hunner “When Python encounters an error in your code, it will print out a traceback. Let's talk about how to use tracebacks to fix our code.” Brian’s commentary Tracebacks can feel like brick wall of error telling you “you suck”. But they are really meant to help you, and do, once you know how to read them. Probably should be one of the earlies

Watch the live stream: Watch on YouTube About the show Sponsored by Datadog: pythonbytes.fm/datadog Brian #1: Python glossary and FAQ Inspired by a tweet by Trey Hunner that referenced the glossary glossary All the Python and programming terms in one place Often refers to other parts of the documentation. Forget what an “abstract base class” is? Just look it up FAQ Has sections on General Python Programming Design and History Library and Extension Extending/Embedding Python on Windows Graphic User Interface “Why is Python Installed on my Computer?” Some decent reading here, actually. Example What is the difference between arguments and parameters? - that’s under Programming Michael #2: Any.io Learned about it via asyncer AnyIO is an asynchronous networking and concurrency library that works on top of either asyncio or trio. It implements trio-like structured concurrency (SC) on top of asyncio Works in harmony with the native SC of trio itself Check out the features AnyIO also comes with i

Watch the live stream: Watch on YouTube About the show Sponsored by us: Check out the courses over at Talk Python And Brian’s book too! Special guest: Matt Kramer (@__matt_kramer__) Michael #1: Survey results Question 1: Question 2: In terms of too long, the “extras” section has started at these times in the last 4 episodes: 39m, 32m, 35m, and 33m ~= 34m on average Brian #2: Modern attrs API attrs overview now focus on using @define History of attrs article: import attrs, by Hynek predecessor was called characteristic. A discussion between Glyph and Hynek in 2015 about where to take the idea. attrs popularity takes off in 2016 after a post by Glyph: ‌The One Python Library Everyone Needs In 2017 people started wanting something like attrs in std library. Thus PEP 557 and dataclasses. Hynek, Eric Smith, and Guido discuss it at PyCon US 2017. dataclasses, with a subset of attrs functionality, was introduced in Python 3.7. Types take off. attrs starts supporting type hints as well,

Watch the live stream: Watch on YouTube About the show Sponsored by us: Check out the courses over at Talk Python And Brian’s book too! Special guest: Kim van Wyk Michael #0: Take our survey: Should we try to shorten the episodes? Please fill out the 3 question Google Form here We’ll be taking a break so see you in two weeks. Also feedback / rate us in your podcast player app Brian #1: Jupyter Games Thorsten Beier “Making their own tiny video games can be a great way for kids to learn programming in a playful matter.” For 2D physics-based games, Box2D, (written in C++), is a 2D rigid body simulation library One Python binding, pyb2d, is from Thorsten Game examples use Ipycanvas, Ipywidgets, and Ipyevents for a place to draw and input events. There are Box2D examples for physics simulations, like internal combustion and a wind tunnel. Game examples, with code, and not that much code billiards Angry Shapes (like Angry birds) World of Goo homage Rocket Color Mixing (it’s oddly satisfying to play

Watch the live stream: Watch on YouTube About the show Sponsored by us: Check out the courses over at Talk Python And Brian’s book too! Special guest: Laís Carvalho Michael #1: Django 4.0 released Django is picking up speed: 4.0 Dec 2021 (+1) 3.0 Dec 2020 (+3) 2.0 Dec 2017 (+7) 1.0.1 May 2010 Feature highlights: The new RedisCache backend provides built-in support for caching with Redis. To ease customization of Forms, Formsets, and ErrorList they are now rendered using the template engine. The Python standard library’s zoneinfo is now the default timezone implementation in Django. scrypt password hasher: The new scrypt password hasher is more secure and recommended over PBKDF2. However, it’s not the default as it requires OpenSSL 1.1+ and more memory. Django 3.2 has reached the end of mainstream support. The final minor bug fix release, 3.2.10, was issued today. Django 3.2 is an LTS release and will receive security and data loss fixes until April 2024. Some backwards incompatible changes you’ll

Watch the live stream: Watch on YouTube About the show Sponsored by us: Check out the courses over at Talk Python And Brian’s book too! Special guest: Leah Cole Brian #1: pytest 7.0.0rc1 Question: Does the new pytest book work with pytest 7? Answer: Yes! I’ve been working with pytest 7 during final review of all code, and many pytest core developers have been technical reviewers of the book. A few changes in pytest 7 are also the result of me writing the 2nd edition and suggesting (and in one case implementing) improvements. Florian Bruhin’s announcement on Twitter “I'm happy to announce that I just released #pytest 7.0.0rc1! After many tricky deprecations, some internal changes, and months of delay due to various issues, it looks like we could finally get a new non-bugfix release this year! (6.2.0 was released in December 2020).” “We invite everyone to test the #pytest prerelease and report any issues - there is a lot that happened, and chances are we broke something we didn't find yet (we bro

Watch the live stream: Watch on YouTube About the show Sponsored by us: Check out the courses over at Talk Python And Brian’s book too! Special guest: Dr. Chelle Gentemann Michael #1: rClone via Mark Pender Not much Python but useful for Python people :) Rclone is a command line program to manage files on cloud storage. Over 40 cloud storage products support rclone including S3 object stores Rclone has powerful cloud equivalents to the unix commands rsync, cp, mv, mount, ls, ncdu, tree, rm, and cat. Brian #2: check-wheel-contents Suggested by several listeners, thank you. “Getting the right files into your wheel is tricky, and sometimes we mess up and publish a wheel containing __pycache__ directories or tests/” usage: check-wheel-contents [[HTML_REMOVED]] [HTML_REMOVED] ex: (venv) $ pwd /Users/okken/projects/cards (venv) $ check-wheel-contents dist dist/cards-1.0.0-py3-none-any.whl: OK Checks - W001 - Wheel contains .pyc/.pyo files - W002 - Wheel contains duplic

See the full show notes for this episode on the website at pythonbytes.fm/260.