-
First go through Chris Hayes' "Python with Stanford Alpaca and Vicuna 13B AI models - A llama-cpp-python Tutorial!"
-
Second, experiment with summarization and title generation for the hall-hoag collection.
-
Assumes
uvis installed: https://docs.astral.sh/uv/getting-started/installation/ -
Assumes https://huggingface.co/eachadea/legacy-ggml-vicuna-13b-4bit/resolve/main/ggml-vicuna-13b-4bit-rev1.bin?download=true is downloaded to
../models/ggml-vicuna-13b-4bit-rev1.bin. That file's about 8GB. -
cd /path/to/llm_chat_summarizer_code/
(I'm leaving out stuff that didn't work because of time passing from when I originally got Hayes' code working.)
Goal: experiment with summarization against the hall-hoag collection.
time uv run ./07_temp_reversion_to_1000_words.py --url "https://url/to/storage/test:2etz4x4h/EXTRACTED_TEXT/"
(This is a dev-server version of this item, but with extracted-text.)
Note: this can take, like, a minute and 30 seconds to run!
[05/Dec/2025 13:47:39] DEBUG [07_temp_reversion_to_1000_words-summarize_text()::54] cleaned_summary, The text describes the actions and beliefs of Dean Acheson, a former Secretary of State in the United States, who is portrayed as being confused about the causes of freedom and serving the interests of Soviet Union and Communism through his work as a lawyer. Acheson was paid by Stalin to represent the Soviet Union and its interests, and he continued to advocate for Soviet Russia throughout his career, even after being appointed to government positions in the US. The text suggests that Aches... (auto-summarization of extracted-text)
Goal: experiment with summarization.
[22/Nov/2023 15:11:06] DEBUG [main04-<module>()::56] output, ``{'choices': [{'finish_reason': 'length',
'index': 0,
'message': {'content': ' The text is a speech given by President '
'Barack Obama after his re-election in '
'2012. In the speech, Obama thanks his '
'supporters and expresses his desire to '
'work with Mitt Romney and their teams to '
'move forward as one nation. He '
'acknowledges that while they may have '
'had differences, they both share a '
'commitment to certain hopes for '
"America's future, such as education, "
'innovation, and a strong military. Obama '
'believes that',
'role': 'assistant'}}],
'created': 1700683764,
'id': 'chatcmpl-898cf652-ce58-4f9f-aaa7-7cd1445b787b',
'model': '../models/ggml-vicuna-13b-4bit-rev1.bin',
'object': 'chat.completion',
'usage': {'completion_tokens': 100,
'prompt_tokens': 1951,
'total_tokens': 2051}}``
Goal: experiment with regular output text.
[22/Nov/2023 14:14:50] DEBUG [main03-<module>()::46] output, ``{'choices': [{'finish_reason': 'length',
'index': 0,
'message': {'content': ' Ada Lovelace was a British '
'mathematician and writer, born in 1815. '
'She is best known for her work on '
"Charles Babbage's early mechanical "
'general-purpose computer, the Analytical '
'Engine. Lovelace is credited with '
'writing the first computer program, '
'which was designed to calculate the '
'Bernoulli numbers. She was also a '
'prolific writer and published several '
'books, including a translation of the '
'Italian novel "The Betrothed"',
'role': 'assistant'}}],
'created': 1700680479,
'id': 'chatcmpl-389e00aa-52c0-4de0-8522-460f076dd81a',
'model': '../models/ggml-vicuna-13b-4bit-rev1.bin',
'object': 'chat.completion',
'usage': {'completion_tokens': 100, 'prompt_tokens': 20, 'total_tokens': 120}}``
Goal: just following video tutorial.
[22/Nov/2023 13:45:48] DEBUG [main02-<module>()::34] text, `` Ada``
[22/Nov/2023 13:45:48] DEBUG [main02-<module>()::34] text, `` Lov``
[22/Nov/2023 13:45:49] DEBUG [main02-<module>()::34] text, ``el``
[22/Nov/2023 13:45:49] DEBUG [main02-<module>()::34] text, ``ace``
[22/Nov/2023 13:45:49] DEBUG [main02-<module>()::34] text, `` was``
[22/Nov/2023 13:45:49] DEBUG [main02-<module>()::34] text, `` an``
[22/Nov/2023 13:45:49] DEBUG [main02-<module>()::34] text, `` English``
[22/Nov/2023 13:45:49] DEBUG [main02-<module>()::34] text, `` math``
[22/Nov/2023 13:45:49] DEBUG [main02-<module>()::34] text, ``ematic``
[22/Nov/2023 13:45:49] DEBUG [main02-<module>()::34] text, ``ian``
[22/Nov/2023 13:45:49] DEBUG [main02-<module>()::34] text, `` and``
[22/Nov/2023 13:45:49] DEBUG [main02-<module>()::34] text, `` writer``
...etc...
Goal: just followign video tutorial.
[22/Nov/2023 12:55:35] DEBUG [main-<module>()::28] output: {
"id": "cmpl-03b92ca8-2d67-4acc-b6f4-2964398e2bd2",
"object": "text_completion",
"created": 1700675728,
"model": "../models/ggml-vicuna-13b-4bit-rev1.bin",
"choices": [
{
"text": "Question: Who is Ada Lovelace? Answer: Ada Lovelace was an English mathematician and writer, born in 1815. She is known for her work on Charles Babbage's early mechanical general-purpose computer, the Analytical Engine. She is considered to be the world's first computer programmer.",
"index": 0,
"logprobs": null,
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 12,
"completion_tokens": 62,
"total_tokens": 74
}
}
python ./main.py 33.89s user 0.72s system 486% cpu 7.117 total
keywords: hall-hoag, summarization