SecondThread's blog

Llama Code: Open Coding LLM
By SecondThread, history, 16 months ago, In English

Llama Code in CP

Today, Meta released Llama Code, a Large Language Model trained specifically for code. It's trained on Python, C++, Java, PHP, Typescript, C#, Bash. They are releasing 3 models with 7B, 13B and 34B parameters respectively. The smaller two allow for Fill-in-the-Middle.

It's free for research and commercial use, but obviously requires a solid amount of computing power if you want to run it locally.

You can read more about it here: https://about.fb.com/news/2023/08/code-llama-ai-for-coding/

I'd be interested in seeing if this becomes useful in CP. It seems like there's a lot of potential, specifically with code completion with Fill-in-the-middle. Imagine having vscode, hitting "F3", typing something like "SegTree with lazy prop, range set max, and range +=", and getting exactly the segment tree you need in two seconds.

I also think it may be useful because although LLMs are currently horrendous at the problem solving aspect of CP (similarly to how tools like MidJourney suck at hands), if you can highlight an area or chunk of code and be like "hey, this part's bad, try again for just this part", that might help improve weaknesses.

Just a thought.

  • Vote: I like it
  • +129
  • Vote: I do not like it

»
16 months ago, # |
  Vote: I like it 0 Vote: I do not like it

This is pretty cool, sounds like codepilot but local

→ Reply
»
16 months ago, # |
  Vote: I like it +147 Vote: I do not like it

wen hacker cup

→ Reply
»
16 months ago, # |
  Vote: I like it 0 Vote: I do not like it

do you think AI will become better than human at CP eventually?

→ Reply
  • »
    »
    16 months ago, # ^ |
      Vote: I like it +13 Vote: I do not like it

    I think it'll be better than the average human within the decade. But (unlike chess) I don't think it'll be better than the best humans in limited-submission, 4+ hour contests until very close to full AGI (or never if we never hit AGI). I can't imagine the world having a scalable entity with that level of problem solving ability, planning, and creativity without imminent runaway risk.

    → Reply
    • »
      »
      »
      16 months ago, # ^ |
        Vote: I like it 0 Vote: I do not like it

      thank god!

      thanks for the reply

      → Reply
    • »
      »
      »
      16 months ago, # ^ |
        Vote: I like it +6 Vote: I do not like it

      SIR what u mean by average human if it in terms of cf rating then according to you what rating do you consider average human ?

      → Reply
»
16 months ago, # |
  Vote: I like it 0 Vote: I do not like it

tools like MidJourney suck at hands

Hands are not an archenemy of AI since inpainting and LoRAs became a thing(so like, for 8 months already), it's just that most people are using such tools to generate slop, and don't care about post production at all.

The current issue with AI generated art is that it looks like play-doh, and models don't really understand the lightning, which makes pictures look even worse. A good eye of a professional will immediately find the inconsistencies, and if you saw a lot of human art, you will see that art feels weird.

I am kinda hyped for what open source community will create, Llama models themselves are a bit mid, but as a baseline, finetuned Llama 1 was almost on par with ChatGPT 3.5

Also, I wonder about your opinion on LLMs used for substituting coders. Artists don't utilize the power of Stable Diffusion or MidJourney to create a good prototype of an image, that they can enhance by hand. Programmers are very different though, they already incorporate ChatGPT in their work a lot. It is very good at writing a lot of mid code, you basically have a power of 10+ coding monkeys. Do people use AI at your job?

→ Reply