Home - News ( United States | United Kingdom | Italy | Germany ) - Football scores

Showing content from https://github.com/vllm-project/vllm/issues/11126 below:

how to use EAGLE on vLLM? · Issue #11126 · vllm-project/vllm · GitHub

The output of `python collect_env.py`

I want to test EAGLE on vllm, but i try so many methods to run EAGLE, fail so many times.
The target model is Llama2-chat-hf, and the draft model is EAGLE-Llama2-chat in original EAGLE's author's github.

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

RetroSearch is an open source project built by @garambo | Open a GitHub Issue

Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo

HTML: 3.2 | Encoding: UTF-8 | Version: 0.7.4