Welcome to Tensors & Quarks
Exploring the cosmos of Physics & the depths of Machine Learning.
Latest Posts
-
From Facts to Insight: Bridging the Compositionality Gap in Language Models
Large language models (LLMs) such as GPT-3 have transformed natural language understanding by memorizing vast amounts of text. Yet, when faced with questions that require combining multiple pieces of knowledge—so-called compositional reasoning—even the biggest models stumble. In their paper Measuring and Narrowing the Compositionality Gap in Language Models, Press et al. introduce a new metric for this shortfall, show that it persists despite model scale, and propose practical prompting techniques to close it.
Read more → -
LoRA: A Breakthrough in Efficient Fine-Tuning of Large Language Models
As large language models (LLMs) like GPT-3, LLaMA, and BERT continue to grow in size and influence, one challenge becomes increasingly apparent: while these models offer exceptional capabilities, adapting them for new tasks remains expensive and resource-intensive. Fine-tuning a model with billions of parameters typically requires large datasets, massive compute power, and hours or even days of training time — luxuries not everyone can afford.
Read more → -
Fine-Tuning Language Models: Welcome to the Nerdy Playground of LLMs
From LoRA to RLHF — and all the acronyms in between
So, you’ve got your hands on a fancy pre-trained language model. Great. It’s read more text than any human ever will, speaks in Shakespearean iambic pentameter and Python, and can tell you the capital of Burkina Faso at 3 AM.
Read more → -
Welcome to Tensors & Quarks
This is the first post!
Read more →
Here I’ll share ideas in physics, AI, and their cosmic overlaps.