• 32 Posts
  • 122 Comments
Joined 3 years ago
Cake day: July 5th, 2023
  • maria [she/her]@lemmy.blahaj.zonetoFree Open-Source Artificial Intelligence@lemmy.world•LLM Assistant for Markdown DocumentsEnglish
    2·
    11 days ago

    what volume of markdown files are we talking?

    also, just so i understand this right:
    you are looking fir a markdown editor which has a chat window on the side which can look at other files to assist in writing.
    is that correct?

    which editor do you use right now for editing the files? does it need to support vim-movements? (if u dont know what that is, it doesnt matter)

    what exactly would the LM be assisting in? should it be to just read files and respond, or edit them itself aswell, or suggest edits?

    suggestion for under 200 files

    depending on the amount of files, a simple index and read-tool functionality might be enough. Here is how you would create such an index:

    1. LM looks at each md file
    2. Then generates a one-liner about the content, like:

    Static functions and variables in Godot, using autoload scripts and scenes (e.g. loading screens, overlays, save systems, anything permanently loaded)

    1. store that into a file alongside each files path, perhaps like this:
    ~/Documents/
  file.md (one-liner here)
  another_file.md (one-liner here)
  topic/
    file_about_topic.md (one-liner here)
[...]

    These three steps can be done using any coding agent you got lying around using this prompt

    Goal: FILE_INDEX.txt which contains all files in <folder with the documents> in tree form (reduce redundant dir paths) with a concise and keyword-heavy one-liner about the files content.

FILE_INDEX.md format:

~/SomeDir/SomeDirBelowIt/
  a_file_in_that_dir.md (description)
  another_file.md (description)
  dir_inside_it/
    file_in_that_dir.md (description)

How: First use `tree` to see the directories contents, then use subagents to delegate the generation of the descriptions. They should out them into files themselves. An agent can summarize 5 files at mist in one go. Tell each agent exactly what name the output file should have. Finally, after having received all descriptions with their file names, combine the files all into one final FILE_INDEX.txt

    This index can then be did into any agent to let it find files quicker, without having to hope for good chunking settings in a RAG pipeline.

    all this was written by a human, even if it might not seem as such.

  • maria [she/her]@lemmy.blahaj.zonetoAI@lemmy.ml•Cohere Drops Command-R 35B 08-2024 Update, Just About a Perfect Local LLM for 24GB GPUs.
    1·
    1 year ago

    i totally agree… with everything. 6GB really is smol and, cuz imma crazy person, i currently try and optimize everything for llama3.2 3B Q4 model so people with even less GB VRAM can use it. i really like the idea of people just having some smollm laying around on their pc and devs being able to use it.

    i really should probably opt for APIs, you’re right. the only API I ever used was Cohere, cuz yea their CR+ model is real nice. but i still wanna use smol models for a smol price if any. imma have a look at the APIs you listed. Never heard of Kobold Horde and Samba so i’ll have a look at those… or i go for the lazy route and chose depseek cuz it’s apparently unreasonably cheap for SOTA perf. so eh…

    also yes! Lemmy really does seem anti AI, and i’m fine with that. i just say yeah companies use it in obviously dum ways but the tech is super interesting which is a reasonable argument i think.

    so yes, local llm go! i wanna get that new top amd gpu once that gets announced. so i’ll be able to run those spicy 32B models. for now i’ll just stick with 8B and 3B cuz they work quick and kinda do what i want.

  • maria [she/her]@lemmy.blahaj.zoneOPtoFree Open-Source Artificial Intelligence@lemmy.world•Before you buy and courses: Read this free prompting guide! (no login required)English
    2·
    1 year ago

    You are completely right and it is mostly about trial and error. I’d assume these courses mainlyl teach things you can do with the big bois, those being by the obvious big evil AI companies. It’s very much an overblown topic and companies pretend it’s actually a fancy thing to learn and be good at.

    The linked guide just explains the basic concepts of few shot prompting, CoT and RAG and stuff. Even these terms I feel, make the topic more complicated than it is. Could literally be summarized to

    • Use examples of what you want
    • Use near-zero temperature for almost everything
    • For complex tasks, tell it to provide its internal thought proccess before providing the answer (or just use the QwQ model)
    • maybe SCREAM AT THE LLM IN ALLCAPS if something is really important