RetroSearch Browse

Home - News ( United States | United Kingdom | Italy | Germany ) - Football scores

Showing content from https://huggingface.co/papers/2306.11644 below:

Website Navigation

Paper page - Textbooks Are All You Need

Textbooks Are All You Need

Published on Jun 20, 2023

· Submitted by akhaliq on Jun 21, 2023 #1 Paper of the day Authors:

,

Abstract

A new compact Transformer-based large language model for code, phi-1, achieves high accuracy on coding benchmarks despite having fewer parameters than competing models.

AI-generated summary

We introduce phi-1, a new large language model for code, with significantly smaller size than competing models: phi-1 is a Transformer-based model with 1.3B parameters, trained for 4 days on 8 A100s, using a selection of ``textbook quality" data from the web (6B tokens) and synthetically generated textbooks and exercises with GPT-3.5 (1B tokens). Despite this small scale, phi-1 attains pass@1 accuracy 50.6% on HumanEval and 55.5% on MBPP. It also displays surprising emergent properties compared to phi-1-base, our model before our finetuning stage on a dataset of coding exercises, and phi-1-small, a smaller model with 350M parameters trained with the same pipeline as phi-1 that still achieves 45% on HumanEval.

Models citing this paper 10 microsoft/phi-1

Text Generation • 1B • Updated Apr 29, 2024 • 9.61k • 213

kenhktsui/llm-data-textbook-quality-fasttext-classifier-v2

Text Classification • Updated 20 days ago • 2.32k • 28

michaelfeil/ct2fast-phi-1

Text Generation • Updated Nov 30, 2023 • 9

OpenNMT/phi-1-ct2-int8

Text Generation • Updated Nov 30, 2023 • 8

Browse 10 models citing this paper Datasets citing this paper 14 HuggingFaceTB/cosmopedia

Viewer • Updated Aug 12, 2024 • 31.1M • 4.58k • 628

maywell/korean_textbooks

Viewer • Updated Jan 10, 2024 • 4.42M • 1.51k • 116