• “Today, Gemini reached an agreement in principle with Genesis Global Capital, LLC (Genesis), Digital Currency Group, Inc. (DCG), and other creditors on a plan that provides a path for Earn users to recover their assets. This agreement was announced in Bankruptcy Court today. This plan is a critical step forward towards a substantial recovery of assets for all Genesis creditors. In addition, Gemini will be contributing up to $100 million more for Earn users as part of the plan, further demonstrating Gemini’s continued commitment to helping Earn users achieve a full recovery.”
• There are 18 EGOT winners.
• YouTubeTV is $65/mo.
• Remember by default, containers build/run as root, and execute processes within as root. You can change these to a specific user for even more control/security.
• Temporary issues with local servers for archive.ubuntu.com:http:* today, apt was installing packages incredibly slowly (or failing).
• Couldn’t find a way to unsubscribe from uber email notifications (they duplicate all the time).
• “Google Invests Almost $400 Million in ChatGPT Rival Anthropic”
• The BMW M1000RR is so sick. 205hp, 420lbs, 4cyl inline, 1000cc, $38k MSRP.
  • For reference: The HP4 was 215hp, 320lbs, 4cyl inline, 1000cc. It was around $85k IIRC. Fully carbon. Not street legal, track only. They only made 750 units in 2017.
• You can nest executions within an f-string, like f"{foo:{bar}}" (eg dynamically padding one variable with spaces according to a width set by another variable).

• Airthings masters kicked off today.
• Champions chess tour page got a rewrite, looks better (much more like chess.com now that it bought chess24/CCT): https://championschesstour.com/schedule/
• They changed the rules *substantially* this year: https://championschesstour.com/format-regulations/
• 6 events the tour finals, each having a play-in and then a knockout. $2m total fund.
• Play-ins are 10+2 9-round swiss, knockouts are 15+3 double match elimination (loser bracket winner must beat winners bracket winner twice to win).
• There are 3 completely separate divisions in knockouts. Div 1 has 8 players and knockout matches are 4 games in the winners bracket and 2 games in the losers bracket. Div 2 has 16 players and knockout matches are 4 games in the winners bracket and 2 games in the losers bracket. Div 3 has 32 players and knockout matches are 2 games in both brackets, with the exception of the final being 4 games.
• Placement in knockout divisions is according to rank in last event’s knockout as well as rank in this event’s play-in. There are weekly qualifiers for 3 non-GMs to get into the play-in.

• SBSC. Switched from uwsgi to gunicorn and moved the proxy infra to ELB.
• https://gitlab.com/bmahlstedt/supercontest/-/issues/186
• Requested a new certificate from ACM, verified via DNS (you have to click the button to create the CNAME records in route53 to prove you own that domain).
• ACM public certs are free. Target groups are free, obviously. The load balancer itself charges on use/traffic.
• Sidenote – accidentally deleted privateer’s cert, had to reissue from amplify (remove-readd domain).
• No longer need the infra containers. nginx-proxy and letsencrypt are gone.
• Python apps speak wsgi (web server gateway interface), of course, but most proxies nowadays can speak this as well (including nginx). They’re not just static file servers nowadays.
• Remember sockets are less overhead than ports.
• uwsgi has a stats server but I don’t use it: https://uwsgi-docs.readthedocs.io/en/latest/StatsServer.html
• You can actually embed your entire app in the uwsgi binary! https://uwsgi-docs.readthedocs.io/en/latest/Embed.html. But still easiest to have the single artifact be a docker image.
• Docker comes with 3 networks predefined that can’t be removed: bridge, host, none.
• Was getting 502 bad gateway in prod. Traced to uwsgi segfaults. Tested and isolated to SC_DEV. In flask debug=False mode, something was breaking with uwsgi. Hence the switch to gunicorn.
• Also from the uwsgi docs: “The project is in maintenance mode (only bugfixes and updates for new languages apis). Do not expect quick answers on github issues and/or pull requests (sorry for that) A big thanks to all of the users and contributors since 2009.” And gunicorn is more popular, more support.
• I ended up still fronting gunicorn with an nginx proxy, but now it’s a simple nginx base image that I copy a custom config in for. No more third-party dependencies on jwilder and letsencrypt.
• SSL is now handled by ELB, which has a cert from ACM attached to the HTTPS listener. You can’t just associate a cert with an EC2 instance. You also can’t copy a cert generated from ACM into your nginx config. You just front with ELB, hence the change. Autorenewal.
• Workers and threads (I’m using gthreads in gunicorn, not sync) are 2*NUM_CORES+1. This is set dynamically, and can read fine in both WSL2 and EC2.
• Remember that although each gunicorn worker starts my flask app in a separate process, instantiating a new apscheduler, they all share a centralized store (no duplicate scheduled jobs).
• Here are the default variables that flask injects into the global jinja context: config, request, session, g, url_for(), get_flashed_messages(). I use all.
• Docker creates a bridge network by default, and exposes it to the internet. The external key is not for connectivity. It just means that the network is managed outside of that compose file.
• expose means it’s available to other containers in docker-compose. ports means it’s available to the host (and therefore anything that can access that host, which for a webserver, is the whole internet).
• Best toplevel docs for nginx: https://nginx.org/en/docs/dirindex.html and https://nginx.org/en/docs/varindex.html
• Nginx returns a 302 redirect for my reverse-proxy-to-gunicorn-container config, so I had to tell AWS that the target group considers both 200 and 302 as successful. Otherwise the health check will declare failure, and the load balancer won’t forward requests to an unhealthy target.
• Full path:
  • Route53 sees the domain request and has an A record to send to ELB.
  • ELB has a listener on HTTPS/443 and an associated SSL cert from ACM.
  • The listener forwards the decrypted requests to a target group configured to receive HTTP/80.
  • The target group forwards to port 80 on the EC2 instance (within the VPC).
  • That EC2 instance is running an nginx container, which forwards 80 from the host to 80 within the container.
  • Nginx proxy passes the traffic to the exposed port 8000 on the gunicorn container.
  • The gunicorn container handles the request (passes to my application, defined in the flask microframework).
• The load balancer is governed by a security group that allows all HTTP/S traffic from all IPv4 addresses.
• The EC2 instance is governed by a security group that allows HTTP traffic from only the load balancer (security group), as well as ssh (from anywhere) for dev.
• Kept an A record in route53 to the elastic IP so that I could ssh to a static address for convenience (my manual work, ansible, etc).
• Remember docker compose will add the name of the services to the network so they’re accessible by other containers in the same network. Eg: My compose file defines supercontest-app, and the nginx container can reach it at http://supercontest-app.
• Health checks to a target group do not count as load balancer capacity units. You’re not charged for these.

• SBSC. Redid all stream management into a proper logging implementation.
• All print() calls gone.
• Every sbsc module logs individually.
• Inspects the logging namespace and handles other loggers too.
• Strips the gunicorn loggers, which have a stream handler which swallows everything, and instead has them propagate up to the root logger (like everything else).
• I then manage the streamhandler and formatter on that, giving full control.

• SBSC. Other.
• Sped up the dockerfiles. It now copies the pyproject toml and poetry lock ONLY into the container before install and the whole source dir, so that docker can cache it and only pip reinstall (the long part) when the lock/toml change.
• Reorganized the dockerfiles and server configs. All are in their appropriate subdirs now, and build contexts are relatively defined.
• Significantly cleaned up the app factory pattern of init.create_app(). It now splits by cli-mode, dev-mode, and prod. Lazy importing only when necessary. Dynamically logging.
• This cleanup fixes a lot. Example: the flask cli runs your factory to access your app. So it prints everything, sets up caching/minification/scheduler/etc – all the stuff it doesn’t need. That’s now fixed.
• This means flask shell and flask db <> are both faster, and don’t log garbage to console.

• Saw wicked on broadway.
• My ranked order of all the ones I’ve seen so far:
  1. Book of Mormon
  2. Moulin Rouge
  3. Harry Potter
  4. Rock of Ages
  5. Lion King
  6. Hadestown
  7. Wicked
  8. Aladdin
  9. Music Man
  10. Six
• Remember intel is arm and apple is amd.
• tail -f /dev/null as docker entrypoint/command to keep container running.
• Properly ran pyenv on wsl2. Installed the python build deps for a clean pyenv python install. Did some cleaning with apt autoremove. Now running python 3.11 on wsl2, macbook, and in all containers running sbsc. For all in both pyenv and poetry env, not system env.
• Installed the Better TOML and Mako vscode extensions for syntax highlighting on those configs.
• Remember that docker down removes volumes as well. So you’ll have to restore the database if you down. Instead, it’s better (in most cases) to just use docker stop.
• poetry export can be used to output a standard requirements.txt from the lockfile.

• SBSC. Upgraded ALL package versions.
• https://gitlab.com/bmahlstedt/supercontest/-/issues/180
• https://gitlab.com/bmahlstedt/supercontest/-/issues/184
• Python 3.8 -> 3.11.
• Flask 1 -> 2.
• SQLAlchemy 1 -> 2.
• Upgraded literally every other package. All flask extensions, all utility libraries, everything. Removed bs4 and decorator. Everything is unpinned in pyproject.toml (locked by lock, of course).
• Removed flask-script and moved over to flask’s (now native) CLI.
• Got flask-monitoringdashboard working again.
• Took advantage of the new py features: enum, walrus operator, switch statements, more.
• Removed the dependence on joyzoursky/python-chromedriver, which stopping uploading a couple years ago (since py3.9). Built my own image, basing from selenium. I then layer pyenv, poetry, and my packages in.
• This was quite the chore. Chrome is not seamless to install (tried to base from py3.11 first). Details on ticket.
• So multiple layers of isolation are now working: host machine -> isolated docker env (with chrome/chromedriver/selenium) -> isolated python env (via pyenv) -> isolated project env (via poetry).
• The client js reimplemented some logic for arePicksOpen() and unstartedStatus/finishedStatuses. These are now ONLY serverside-defined. Single source of truth.
• Exposed week/season as an arg for the scrapers to test them outside the current week. Looks good for next season.
• Selenium has a cool element.submit() which walks up the dom and clicks the form’s submit button. Better than finding the el by xpath/whatever and .click(), which has “Element is not clickable” errors sometimes (viewpoint, scroll, elements over it, etc).
• flask-user is yanked. The maintainer is no longer developing on it, but I’m not sure why the yank. flask-security was forked in 2020 and is maintained by someone else now. flask-login doesn’t have everything I want. So I just kept flask-user for now.
• Made some lint and style improvements, corresponding to newer versions of python/stdlib/pylint/black.
• Fixed all logos in the stack description. Got off clearbit. Made the readmes linked, so you don’t have to keep 2 sources of truth.
• Wrote some super helpful new gnumake targets to drop into various shells with various contexts (active flask app, env vars, etc). This organized things a bit better too; docker compose is now only used for targets that manage container state. Everything else just uses docker.
• Note: upgrading the version of flask-migrate works on its own if you don’t already have a migrations/ folder. If you do, the site-packages update is not sufficient; your local env.py and alembic.ini and script.py.mako will be out of date. I updated them manually from github src.
• Moved all constants into supercontants.core.constants. Much better now.
• Cleaned the explicit template argpassing, jinja globals, and jinja exposing vars to js.
• Changed DB statuses to new format.
• Linted all js.
• Made version management of the sbsc package (itself) single-source-of-truth, instead of duplicating for sentry-sdk and flask-monitoringdashboard. Now, just pypoetry.toml need bumping.
• Cleaned some custom functions, just used configparser from stdlib.

Python version history comparison today. Scraped the releases and decided what version to run my apps on, and what new features to add to them.

3.0 (2008-12-03)

• print statement -> function
• dict.iteritems() -> dict.items()
• raw_input() -> input()
• many things return iterators instead of lists (zip, map, keys, etc)
• unicode string no longer necessary, assumes it
• string formatting, starting to deprecate % and use format
• using with for context management

3.1 (2009-06-27)

• collections.OrderedDict added
• collections.Counter added
• unittest can be marked skip and expectedFailure
• .format() can automatically add commas in the thousands places

3.2 (2011-02-20)

• Stable ABI (easier to write extensions across versions)
• argparse replaced optparse, which only had support for positional args
• logging config by json file
• concurrent.futures for thread/process management
• pyc moved to __pycache__
• wsgi headers need to be native strings, no encoding
• @functools.lru_cache added
• itertools.accumulate() added
• contextlib now allows contextmanagers to be used as function decorators as well
• can now use a .pdbrc

3.3 (2012-09-29)

• added venv as a stdlib module, for programmatic access from python, as well as command line access via pyvenv (and then python -m venv)
• added mock to unittest in stdlib
• added py launcher for windows (double clicking .py automatically runs them)
• yield from to delegate generators
• __qualname__ for fully qualified path to that object

3.4 (2014-03-16)

• asyncio added
• enum added
• pathlib added
• statistics added
• tracemalloc added
• pip is shipped with python now, always

3.5 (2015-09-13)

• typing (could use comments and annotation before, but this formalizes)
• coroutines can be used with async / await syntax (diff syntax before)
• can multiply matrices with A @ B instead of dot(A, B)
• much of stdlib reimplemented in C, improving perf significantly
• os.scandir() added (better than the os.walk function)
• math.inf and math.nan constants were added

3.6 (2016-12-23)

• f strings
• dicts use 25% less mem
• underscores in numbers
• async generators and async comprehensions
• typing for variable annotations
• secrets module added to stdlib
• asyncio was introduced in 3.4 as provisional but is now declared stable

3.7 (2018-06-27)

• types now have postponed evaluation; basically annotations allow hoisting
• breakpoint() added (internally, this calls sys.breakpointhook(), which calls pdb.set_trace())
• the time module now supports nanoseconds (for most functions)
• typing was introduced in 3.5 as provisional but is now declared stable
• added dataclasses

3.8 (2019-10-14)

• walrus operator
• / in function param means it MUST be positional (can’t be keyword)
• using = in an fstring will insert not just the value of the var but also the name of the var (really helpful for printing and debugging)
• async mocks were added to unittest

3.9 (2020-10-05)

• dict merge | (same as {**d1, **d2}) and update |= (same as dict.update) operators
• graphlib added to stdlib

3.10 (2021-10-04)

• much better error messages in standard exceptions (more specific syntax)
• structural pattern matching (switch statements!)
• type union operator: you can do int | float instead of Union[int, float]
• distutils is deprecated, and will be removed in 3.12 (just setuptools now)

3.11 (2022-10-24)

• all around 10-60% faster, average 1.25x, uses slightly more mem tho
• added tomllib, can parse toml now with stdlib
• even more specificity in tracebacks, pointing to exact problem
• multiple exceptions can be raised simultaneously in groups
• ton of deprecated stdlib modules and functions removed (none that I really use)

• ChatGPTplus will be $20/mo: https://openai.com/blog/chatgpt-plus
• Here’s Sapolsky’s lecture series that matches much of the content in Behave: https://www.youtube.com/watch?v=NNnIGh9g6fA&list=PL848F2368C90DDC3D
• Karpathy’s neural networks course is excellent, I’ve heard: https://karpathy.ai/zero-to-hero.html (will do eventually)
• MS confirmed that any funds deposited into the IRA account will be automatically allocated by the manager. At 50k, the self-managers open up, of course.
• SBSC. Switched supercontest-client over to poetry.
  • https://gitlab.com/bmahlstedt/supercontest/-/issues/182
  • Seamless overall. Generate API key in pypi, add to poetry config, build and publish.
  • Yanked the old packages under the name supercontest. It’s supercontest-client now.
• Random Python reminders.
  • Will always be duck typed (dynamic). But you can utilize type hints to make it closer to static typing as much as you’d like. Python does not check these type hints at runtime. It does not care. They are simply used by static analysis tools to flag issues before running.
  • pysa is meta’s static analysis tool for python. Does some linting, style, and security checks. Runs well with pyre, their type checker.
  • Switch statements are the declarative version of a bunch of if statements, which would be imperative. Much clearer as switch. Happy python added this in 3.10.
  • Remember that most comparisons are equality comparisons (for literals). For singletons like True/False/None, it’s an identity comparison.
  • For poetry, the caret ^ in version specification means anything up to the next major version (technically the leftmost number, which in semver in always major).
    • ^2.0.0 = ~2 = 2.*
  • The __future__ module is python’s way of introducing incompatible changes (outside of semver).
  • Remember asyncio is another option for concurrency. Multiprocessing is … multiple processes. Separate. Understandable. Threads are great, but python’s GIL prevents them from being truly concurrent. Coroutines are the answer. Asyncio allows you to get around the GIL in a single thread. Call an async function, don’t wait for it to return.
    • And then greenlet is similar to threads, but they don’t let the OS scheduler handle switching. You write in the switch points into your program logic. It’s explicit in your code.
  • The stdlib email module is not an all-in-one mail solution, it’s mostly just for the message handling (not the sending – that’s the smtplib builtin module).
    • flask-mail just wraps those two. It’s literally one file: https://github.com/mattupstate/flask-mail/blob/master/flask_mail.py
  • The stdlib venv module obviously ships with more recent python versions. It acts like virtualenv, just call python -m venv instead. But this is rarely needed anymore; use pyenv to manage system python environments and poetry to manage virtual environments. There are higher-level tools to manage executables and site-packages by project/image/path/machine nowadays.
• Looked up trade-in value for my old dell xps13, lenovo thinkpad, and ipad. Combined, it would be around $20, so not worth the time. I’ll probably just recycle them.
• Removed ipad from apple account, find my *, icloud, etc.
• Unicode 15 is the latest release. 149,186 characters.

• Shot the podcast.
• Spent the day in wynwood and south beach.
• Lots of private work.
• Brady’s Fox analyst deal is 10yr/$375m.
• Signed out of appleID on mac and back in to resolve the incessant “iphone wants to access this macbook – approve/deny?”
• Asked MS to allocate the deductible 2022 6k and 2023 6.5k contributions from cash. into equities (same distribution).
• My building is reopening the garage on march 1st. I asked if I can bring the ducati back the night before, feb 28th (motorgrrl charges on the 1st of the month, for the whole month).
• Earnings.
  • AAPL: miss 3.64% eps, miss 3.7% rev.
  • GOOG: miss 12.21% eps, miss 0.58% rev.
  • AMZN: miss 82.6% eps, beat 2.35% rev.
  • META: miss 20.8% eps, beat 1.54% rev.
• My verizon autopay uses my boa debit because it provides a 2% discount. This is given for all debit autopay, and provides the 2% return in Verizon Dollars. These are only usable for verizon things. I switched it over to CSR, which has higher rewards, and can be used universally.
• To delete columns of text in vscode like vim, use ctrl-alt-mouseDrag.
• Lots of work on the SBSC ticket to upgrade python, flask, and many other versions. Ended up getting sidelined writing my own image for chrome/chromedriver/python/selenium. Will add details once complete.
• While the macbook air does have a dedicated magsafe charging port, it can also charge via usb-c.

• Dymicron now makes diamond discs for replacement surgery: https://invest.dymicron.com/
• Delta is turning on free (viasat) wifi today for most of its planes.
• They’re not just bringing back the mammoth – they’re reintegrating the dodo as well!
• Lots of private work.
• Flew to miami.
• Transferred tender disbursement to TD and prepared positions.
• Added boa to ms and transferred IRA contributions. Since traditional, not roth (contribute after-tax), can deduct the full contribution. Did the maximums, 2022 individual 6k and 2023 individual 6.5k.
• Macbook.
  • Changed default shell to bash. (chsh -s /bin/bash)
  • Copied my dotfiles over.
  • Ended up removing those. I’ll just use z shell, good chance to familiarize anyway.
  • Added a couple aliases.
  • Generated ssh keypairs for various things.
  • Installed homebrew.
  • Installed the_silver_searcher (ag).
  • Installed poetry.
  • Installed rosetta 2.
  • Installed docker (docker desktop for mac, apple silicon).
  • Installed ansible (comes with ansible-playbook).
  • Was able to get into the prod EC2.
  • Did a prod db backup successfully (remotely).
  • By default on new macbooks, Airplay Receiver is listening on port 5000. Disable this in system settings so that you can run your flask apps.
  • Successfully got sbsc up locally.
• Frontier charges extra for a carry-on ($65), and it’s more than checked baggage for most other airlines.
• The compose utility now comes with docker.
• The strassburg sock is the one for plantar fasciitis that flexes your foot all night.
• SBSC.
  • https://gitlab.com/bmahlstedt/supercontest/-/issues/181
  • Linted the whole project. Few other little improvements as well.
  • Generators are faster, use them for iteration ranges if you don’t need to do anything with the result list (index, fetch item, etc). This just looks like tuple comprehension for literal syntax (list comprehension but replace brackets with parentheses).
  • Remember to use custom sqlalchemy expressions for boolean comparison (like pylint singleton-comparison, can’t just do “column is True” in an ORM query).
  • Switched to the application factory model for things like sqlalchemy/mail/scheduler/etc instantiation. Previously these objects were instantiated in the main app init. Then other modules would import them when they needed them (like models importing db, the sqlalchemy object). This creates a ton of cyclic imports. Instead, just instantiate the objects in the submodules and have the main app init.create_app import them. Then you can do some cool stuff like create multiple apps, all using the same extensions. Or different test harnesses.