Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
95 commits
Select commit Hold shift + click to select a range
d91fd7c
fixed issue: crashed on deleting chat
rifkybujana Jan 15, 2025
e5975a2
- Added persistence KV Cache method\
rifkybujana Jan 22, 2025
4728dbf
added qwen 2.5 code 0.5b - 14b, and added qwen 2.5 14b models
rifkybujana Jan 22, 2025
6a78b30
Merge branch 'main' into dev
rifkybujana Jan 22, 2025
74a586f
merge with main
rifkybujana Jan 22, 2025
8daed41
fix model won't generate if we switch the model on a non empty chat
rifkybujana Jan 22, 2025
ba9446c
added deepseek r1
rifkybujana Jan 23, 2025
535bd81
removed deepseek unsupported
rifkybujana Jan 23, 2025
812cd45
Revert "added deepseek r0"
rifkybujana Jan 23, 2025
ccfc310
Revert "removed deepseek unsupported"
rifkybujana Jan 23, 2025
69268ad
added deepseek r1 support
rifkybujana Jan 23, 2025
517bd1f
added base markdown rendering
rifkybujana Jan 26, 2025
4135ddf
added ImGuiColorTextEdit for handling code rendering on markdown
rifkybujana Jan 26, 2025
847c726
added markdown renderer
rifkybujana Jan 28, 2025
e9e827a
added modified imgui_md
rifkybujana Jan 28, 2025
66c18f2
added cancel button and fix the model card duplication issue
rifkybujana Feb 1, 2025
e8bde60
fix last selected model issue
rifkybujana Feb 1, 2025
60b686e
added tps
rifkybujana Feb 1, 2025
c1ba124
don't show empty thought
rifkybujana Feb 1, 2025
899a8c2
add automation to detect number of thread to use
rifkybujana Feb 1, 2025
67c2a3a
add fallback to failed model loading
rifkybujana Feb 1, 2025
c813e6b
Merge branch 'main' into dev
rifkybujana Feb 1, 2025
42fb04a
fix model loaded syncronization on start and code rendering block glitch
rifkybujana Feb 1, 2025
d73ab5e
remove text debugging
rifkybujana Feb 1, 2025
dbfb365
refactored history sidebar
rifkybujana Feb 5, 2025
65f45b2
refactored preset sidebar
rifkybujana Feb 5, 2025
5d53b90
fix preset selection
rifkybujana Feb 7, 2025
0ea6802
re-refactor chat history sidebar
rifkybujana Feb 7, 2025
c00e99f
refactor model manager
rifkybujana Feb 7, 2025
83c12cd
rename constant namespace
rifkybujana Feb 7, 2025
4a25059
refactor chat history render
rifkybujana Feb 7, 2025
c00d90a
refactor chat_section
rifkybujana Feb 7, 2025
b58f1db
refactor main code and fixed preset selection duplication bug
rifkybujana Feb 8, 2025
90ebc27
fixed kv cache deletion and renaming
rifkybujana Feb 8, 2025
090a986
moved tab manager to ui/tab_manager.hpp
rifkybujana Feb 8, 2025
36af3a1
added stop generation button
rifkybujana Feb 10, 2025
ea59aaa
refactor input field rendering
rifkybujana Feb 10, 2025
73d2e71
added regenerate button
rifkybujana Feb 10, 2025
141aaf6
added regenerate functionality
rifkybujana Feb 10, 2025
88f0b90
stop all jobs on exit
rifkybujana Feb 10, 2025
8a20091
track job ids within model manager
rifkybujana Feb 10, 2025
d564491
restyle loading bar
rifkybujana Feb 10, 2025
9a5bfa8
[workaround] fixed the delete model button didn't work
rifkybujana Feb 10, 2025
19d35c1
handle model loading asynchronously
rifkybujana Feb 11, 2025
d1b6d30
refactored progress bar widget
rifkybujana Feb 11, 2025
8d31c98
fixed can't find IndeterminedProgressBar error, and progress bar pos…
rifkybujana Feb 11, 2025
abeda36
add unload model functionality
rifkybujana Feb 12, 2025
c1d4967
fixed chat code block ui glitch
rifkybujana Feb 12, 2025
88f38f5
fixed bug model trying to regenerate even if no model loaded, and add…
rifkybujana Feb 12, 2025
7c9c368
added context shifting on the engine
rifkybujana Feb 13, 2025
67bcfc7
if max_new_token set to be 0, don't stop until eos (infinitely generate)
rifkybujana Feb 13, 2025
8df4d2c
update engine
rifkybujana Feb 13, 2025
626967e
update the kv cache loading for context shifting
rifkybujana Feb 13, 2025
ac70696
Merge branch 'main' into dev
rifkybujana Feb 13, 2025
c09aa85
fix delete chat doesn't have to pass model name and variant name
rifkybujana Feb 13, 2025
9b6f5f0
Merge branch 'dev' of https://github.com/genta-technology/kolosal int…
rifkybujana Feb 13, 2025
bdd81f3
fixed merge with main branch
rifkybujana Feb 13, 2025
9139b34
Merge branch 'main' into dev
rifkybujana Feb 13, 2025
5eb446d
refactor model generation callback
rifkybujana Mar 2, 2025
959fcd1
added kolosal server library
rifkybujana Mar 3, 2025
6456ce7
model server tab
rifkybujana Mar 5, 2025
c49bb59
added server logs, model selection, tab selection buttons, and loadin…
rifkybujana Mar 6, 2025
f975644
added server functionality
rifkybujana Mar 7, 2025
96ce0d4
added reload model buttons
rifkybujana Mar 7, 2025
0b6eb1c
update installer version
rifkybujana Mar 7, 2025
806d6dd
fixed kv cache sequence id
rifkybujana Mar 8, 2025
8e5d4c4
merge with main
rifkybujana Mar 8, 2025
f59c89e
fixed seq id pass for server
rifkybujana Mar 9, 2025
4a0b385
acrylic window
rifkybujana Mar 13, 2025
05f57d0
refactor ai model config
rifkybujana Mar 13, 2025
37cc8df
added section of downloaded models
rifkybujana Mar 13, 2025
cc15d63
sort model based on alphabet
rifkybujana Mar 13, 2025
6ecc1e8
added search in model manager modal
rifkybujana Mar 13, 2025
d1ac4e2
fixed pasted a very long text into system prompt, crashed the applica…
rifkybujana Mar 14, 2025
36179e8
fixed chat name can't contain some symbols
rifkybujana Mar 14, 2025
55706e0
allowing to rename duplicate chat name
rifkybujana Mar 14, 2025
4b082d7
automatically adjust value for n_batch and n_keep to be within the n_…
rifkybujana Mar 14, 2025
928b35b
update installer script
rifkybujana Mar 15, 2025
05d3167
merge with main
rifkybujana Mar 15, 2025
f340e9a
added fallback if the windows' transperancy effect is disabled in the…
rifkybujana Mar 15, 2025
b865aab
fixed url opening through markdown
rifkybujana Mar 16, 2025
9360175
added system prompt editor
rifkybujana Mar 16, 2025
46bbab1
added auto scroll
rifkybujana Mar 17, 2025
e86c623
fixed height model variant scroll
rifkybujana Mar 17, 2025
0d9e11e
added model size in gb
rifkybujana Mar 20, 2025
986e21e
added status bar footer
rifkybujana Mar 21, 2025
9f2ac52
fix layout to add scrollbar in chat window
rifkybujana Mar 21, 2025
2199a6b
added /completion endpoint
rifkybujana Mar 24, 2025
c5a2444
added multi model and smart memory approximator
rifkybujana Apr 6, 2025
8bf12e5
added threadpool to model manager and fixed gpu memory usage
rifkybujana Apr 8, 2025
8c47b4b
use vulkan backend detector
rifkybujana Apr 8, 2025
305b784
fixed gpu vram monitoring to use directx
rifkybujana Apr 8, 2025
4472c4b
fixed crashed on multimodel by threadpool
rifkybujana Apr 9, 2025
89f0d1b
update nsis installer
rifkybujana Apr 9, 2025
589043d
merged with main
rifkybujana Apr 9, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 1 addition & 11 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -24,13 +24,6 @@ option(DEBUG "Build with debugging information" OFF)

# ==== External Dependencies ====

# OpenGL
find_package(OpenGL REQUIRED)
if(NOT OpenGL_FOUND)
message(FATAL_ERROR "OpenGL not found")
endif()
message(STATUS "Found OpenGL: ${OpenGL_INCLUDE_DIR}")

# OpenSSL
find_package(OpenSSL REQUIRED)
if(NOT OpenSSL_FOUND)
Expand Down Expand Up @@ -112,7 +105,6 @@ target_compile_definitions(kolosal_lib PUBLIC
target_include_directories(kolosal_lib PUBLIC
${IMGUI_DIR}
${IMGUI_DIR}/backends
${OpenGL_INCLUDE_DIR}
${EXTERNAL_DIR}/glad/include
${EXTERNAL_DIR}/icons
${EXTERNAL_DIR}/nlohmann
Expand All @@ -131,7 +123,6 @@ target_include_directories(kolosal_lib PUBLIC
# Platform-specific library dependencies
if(WIN32)
target_link_libraries(kolosal_lib PUBLIC
${OpenGL_LIBRARIES}
glad
nfd
Dwmapi
Expand All @@ -153,12 +144,11 @@ if(WIN32)
)
else()
target_link_libraries(kolosal_lib PUBLIC
${OpenGL_LIBRARIES}
glad
nfd
OpenSSL::SSL
)

target_compile_definitions(kolosal_lib PUBLIC
IMGUI_IMPL_OPENGL_LOADER_GLAD
${FONT_DEFINITIONS}
Expand Down
Binary file modified external/genta-personal/bin/InferenceEngineLib.dll
Binary file not shown.
Binary file modified external/genta-personal/bin/InferenceEngineLibVulkan.dll
Binary file not shown.
2 changes: 1 addition & 1 deletion external/genta-personal/include/types.h
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ struct LoadingParameters
bool warmup = false;
int n_parallel = 1;
int n_gpu_layers = 100;
int n_batch = 4096;
int n_batch = 256;
};

#endif // TYPES_H
2 changes: 2 additions & 0 deletions include/config.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

#include <imgui.h>

#define APP_VERSION "0.1.8"

// TODO: Need to refactor this to use json file that is modifiable by the user in realtime
// Set up a system to save and load the settings from a json file
Expand Down Expand Up @@ -132,4 +133,5 @@ namespace Config
constexpr float INPUT_HEIGHT = 100.0F;
constexpr float CHAT_WINDOW_CONTENT_WIDTH = 750.0F;
constexpr float TITLE_BAR_HEIGHT = 50.0F;
constexpr float FOOTER_HEIGHT = 22.0F;
} // namespace Config
Loading
Loading