7 minutes of Mira Murati

A great, great introduction to LLMs and transformers — and a chance to ogle at OpenAI’s CTO!

Covering attention, tokens, RLHF etc in 7 minutes enough for most.

Persistent Python

Nicholas Carlini’s great addition to Python DS stack — an environment state freezing machine!

Briefly, it’s a ~100 line script that allows me to snapshot the Python environment at an arbitrary point in time, and then interactively make changes to the code and run different experiments on the current state.

Paper release checklist

Again, Nicholas at it with a great checklist covering re-checking paper title to short scripts for e.g.

[x for x in open("/tmp/paper.tex") if 'TODO' in x or 'FIXME' in x]

Tips, cheatsheets and sheets

An awesome list of tools, cheatsheets, awesome lists:

• CLI Tools
• GUI Tools
• Web Tools
• Systems/Services
• Networks

• Containers/Orchestration
• Manuals/Howtos/Tutorials
• Inspiring Lists
• Blogs/Podcasts/Videos
• Hacking/Penetration Testing
• Your daily knowledge and news
• Other Cheat Sheets
• Shell One-liners
• Shell Tricks
• Shell Functions

Generating generators + another article

You can iterate, or you can generate for:

• a kind of lazy evaluation and memory savings (you produce the next item only when needed/requested)
• producing an arbitrary number of items as opposed to generators like iterate()
• (citing) automatically implementing __iter__(), __next__(), and StopIteration which otherwise, need to be explicitly specified

Coroutines

Another iteration type that can run asynchronously:

async def factorial(n):
    if n == 0:
        return 1
    else:
        return n * await factorial(n - 1)

async def main():
    result = await factorial(10)
    print(result)

asyncio.run(main())

Welcome to Teleogenic/Boi Diaries❣️

Other places I cross-post to:

• Hashnode
• Medium
• Telegram
• Twitter
• LinkedIn

Reuse

MIT

Citation

BibTeX citation:

@online{kogan2023,
  author = {Kogan, Zakhar},
  title = {Murati, Persistent {Python} and Tips},
  date = {2023-09-17},
  url = {https://teleogenic.com/posts/230910-murati},
  langid = {en}
}

For attribution, please cite this work as:

Kogan, Zakhar. 2023. “Murati, Persistent Python and Tips.” September 17, 2023. https://teleogenic.com/posts/230910-murati.

[object Object]