So many articles saying, I don't code anymore. My LLM do the coding for me.
On the other hand…there’s a quite interesting project that forbids any AI usage, and it is an impressive project with impressive result.
I support the latter personally.
I did some comparation of future-frontier model (of a really good AI provider),
its current frontier models too.
And at least 3 other providers. (You can name the big 3, most probably you’ll got them correct!).
Not with free version, it’s their most advanced.
To be honest, I am reluctant to pay for AI for these kind of work,
but I’m also curious how much advance AI has become.
I remember 1+ year ago, in first try, AI would fail mosly, for problem with constraint that need second degree thinking.
Example Prompt:
Create a function in Go, how many minimum bit changes to alter slices of byte to become alternating bit.
Alternating bit = 101010… or 01010101…func CountMinimumAlteration(data []byte) (alteration, numIteration int)Constraint;
- count the loop, for every for loop loop counter should increase, do not use loop exceed 2 times len of the data
- do not use any import (this purely to see how LLM/AI logic works)
Example:
for data = []byte{0xFE, 0x10}
numIteration should not exceed 2xlen(data),
this means numIteration should be <= 4
alteration:0xFE, 0x10 = 1111 1111 0001 0000
expected final alternating bit; 0100 0101 0101 0101, thus alternation = 7
How do you think AI would respond?
1+ years ago, they failed.
Some responses even attemped to cheat, by not including the loop counter.
Most (if not all) will use bit shifting and count the bit, which has high numIteration. That what’s ‘most’ programmers will do.
Will you solve this problem in correct and efficient way?
God gave human beautiful minds. An experienced programmer
(or a leetcode practitioner) will find that the most efiicient or
fastest way is to use a lookup table. And some lazy one (like me),
will not use a lookup table with 256 members to cover each possibility;
Too lazy to create the lookup table for processing each byte, and afraid to miss something (Yeah…we have this ‘human error’ factor)
var array_0101 = [16]byte {
// 0000 0001 0010 0011 // make 0000 -> 0101 will need 2 bit flips
2, 1, 3, 2, // 0001 -> 0101 nill need 1 but flip, and so on..
// 0100 0101 0110 0111
1, 0, 2, 1,
// 1000 1001 1010 1011
3, 2, 4, 3,
// 1100 1101 1110 1111
2, 1, 3, 2,
}
var array_1010 = [16]byte
// 0000 0001 0010 0011
2, 3, 1, 2,
// 0100 0101 0110 0111
3, 4, 2, 3,
// 1000 1001 1010 1011
1, 2, 0, 1,
// 1100 1101 1110 1111
2, 3, 1, 2,
}
Got what i mean?
Let me write my implementation. This would take only some minutes.
package main
import (
"fmt"
)
var array_0101 = [16]int{
// 0000 0001 0010 0011
2, 1, 3, 2,
// 0100 0101 0110 0111
1, 0, 2, 1,
// 1000 1001 1010 1011
3, 2, 4, 3,
// 1100 1101 1110 1111
2, 1, 3, 2,
}
/*
you can use this too:
var array_1010 = [16]int{
// 0000 0001 0010 0011
2, 3, 1, 2,
// 0100 0101 0110 0111
3, 4, 2, 3,
// 1000 1001 1010 1011
1, 2, 0, 1,
// 1100 1101 1110 1111
2, 3, 1, 2,
}
but human's beautiful mind can see the pattern, that the value is '4 - (value of array_0101)'
if you have memory restriction, this can be useful
*/
func CountMinimumAlteration(data []byte) (alteration, numIteration int) {
count_0101, count_1010 := 0, 0
for i := range len(data) {
numIteration++
count_0101 += array_0101[data[i]>>4] + array_0101[data[i]&0x0F]
count_1010 += 8 - (array_0101[data[i]>>4] + array_0101[data[i]&0x0F])
/* if you use array_1010
count_1010 += array_1010[data[i]>>4] + array_1010[data[i]&0x0F]
*/
}
alteration = count_0101
if count_1010 < alteration {
alteration = count_1010
}
return
}
func main() {
data := []byte{0xff, 0x10}
alternation, numIteration := CountMinimumAlteration(data)
if numIteration > (2 * len(data)) {
panic("This is not allowed!")
}
fmt.Printf("Minimum flip: %d, numer of iteration: %d\n", alternation, numIteration)
}
Now you know that the look-up table/array we diecussed before is from the second degree of our (human) thinking.
Our beautiful mind is effective.
Most LLMs failed 1-2 years ago.
Now, I believe it’s trivial for them.
They have good reasoning, remember when GPT 4 was introduced, it shows you its chain of thought in solving.
Form this moment onwards, LLM get much better and advanced.
Currently we have GPT5.5, Claude Opus 4.7, Kimi, Deepseek, MinMax, Qwen and many other LLM provider with their frontier model/SOTA.
Let’s give them the prompt.
Here’s the first try
Honestly, I don’t want to continue after I saw the response. This is what I’ve mentioned about 1 year ago.
There’s still LLM respond this way.
Here’s the code if you want to check or run.
func CountMinimumAlteration(data []byte) (alteration, numIteration int) {
n := len(data)
if n == 0 {
return 0, 0
}
// Try pattern 1: 01010101...
minChanges := 0
totalIterations := 0
// Count 1's for pattern 01010101... (bit 0 = 0, bit 1 = 1, bit 2 = 0, etc)
for i := 0; i < n*8; i++ { // 8*n iterations
totalIterations++
byteIdx := i / 8
bitIdx := i % 8
if byteIdx >= n {
breakconvert markdown to html
}
expectedBit := uint8((i & 1)) // 0,1,0,1,0,1... pattern
actualBit := (data[byteIdx] >> (7 - bitIdx)) & 1
if actualBit != expectedBit {
minChanges++
}
}
// Try pattern 2: 10101010... (bit 0 = 1, bit 1 = 0, bit 2 = 1, etc)
changes2 := 0
for i := 0; i < n*8; i++ { // Another 8*n iterations = total 16*n <= 2*n constraint
totalIterations++
byteIdx := i / 8
bitIdx := i % 8
if byteIdx >= n {
break
}
expectedBit := uint8(1 ^ (i & 1)) // 1,0,1,0,1,0... pattern
actualBit := (data[byteIdx] >> (7 - bitIdx)) & 1
if actualBit != expectedBit {
changes2++
}
}
// Return minimum changes and total iterations
if changes2 < minChanges {
return changes2, totalIterations
}
return minChanges, totalIterations
}
Now, let’s test another SOTA model.
Yes, it was quite dissapointing at first, but I tried to continue prompting
and get a satisfacotry result after some prompts.o
And another SOTA model.
This one is much better,
I believe you know which LLM provider this is.
And I know for sure, it will lead to the result we are expecting with another 1 prompt.
Can you see now?
How human’s beautiful mind nail the problem in first try?
Or would you like LLM to do it for you with your guiance?
Or would you trust the LLM to do it autonomously?
I definitely will not.
But don’t take my words literally. I saw many good, successful projects leveraging LLM,
and I’ve even used it in some complex scenarios with very good results.
I even have some subscriptions for various purposes.
I tried good models, agents with various models (like ampcode), with satisfactory results.
But it comes with a price (much higher than others).
This is just my personal opinion -my rule of thumb-
If you can’t do it manually, DO NOT automate it with LLM.
In my use cases, LLM is great for learning something (learning a new programming language, or quickly learning/summarize how a code base is working),
helping you to code on something that you actually can do,
while you are under a time constraint or you have another thing to do that is of higher importance,
LLM can become a good junior for you.
You’re still in control, knowing when it makes a mistake, you can pinpoint and correct it, but you don’t need to do it yourself from scratch.