No Comments

Open source coding using your Raspberry Pi as a CoPilot/AI assistant in ZED

February 2nd, 2025

In this blog, we’ll delve into the benefits of using AI in the open source coding-app Zed and explore how it can help to write better and faster code with the assistance of a Raspberry Pi.

For AI we don’t use ollama, but we run Llamafile on the Raspberry Pi.

Llamafile is created by, to make LLMs much more accessible to both developers and end users. Also it’s often faster than Ollama, and had a more performant server. That’s important running a LLM on a rather low-end Raspberry Pi 5.

So ssh into you’re Pi5 and startup Llamafile:

dev/llamafile/Llama-3.2-1B-Instruct.Q6_K.llamafile --server --v2 -l 0.0.0.0

To make ZED use your Pi, add this to your settings file CTRL+,:

The important thing is to set the name to openai, and add a api_url in which you set your local endpoint.That way it will use the openai API on a local endpoint instead of the cloud.

“language_models”: {
“openai”: {
“api_url”: “http://raspberrypi.local:8080/v1”
}
},
“assistant”: {
“version”: “2”,
“provider”: “openai”,
“type”: “openai”,
“default_model”: {
“provider”: “openai”,
“model”: “rpi”
}
}

For the rest just add a bogus key in the Assistant configuration for ZED openai, and you’re ready to go.

Llama-3.2 is running fine on a Raspberry Pi 5, it’s a good tradeoff between quality and speed for lower-end hardware. You could also try Qwencoder2.5, that is a bit slower though.

Just run another Llamafile on your Pi, there is no need to reconfigure the settings in ZED. That’s is the fun and easy of this solution.

No Comments

Yes, Google did it, they went REALLY UGLY with forcing JavaScript

January 26th, 2025

About 10 years ago, we warned about the danger that the internet will become a privacy unfriendly place, due to bad behavior of Big Tech.

We more or less predicted that time, that Javascript would become required.

Breaking the bad, pushing a worse internet

Breaking the bad, pushing a worse internet (part II)

And now in 2025, 10 years later, Google just did that.

So when you visit an URL you get zilch info, a blank page, you need to give Google a permission to execute unknown code on your computer to see any content. That is of course breaking the internet. What is the use of an url when there is no content?

https://www.google.com/search?hl=en&q=why%20is%20Javascript%20required

<noscript><style>table,div,span,p{display:none}</style>
 <meta content="0;url=/httpservice/retry/enablejs?sei=xXXX_XXXxx" http-equiv="refresh">
 <div style="display:block">Please click <a href="/httpservice/retry/enablejs?sei=xXXX_XXXxx">here</a> if you are not redirected within a few seconds.</div>
</noscript>

 

Yes there is a collection of scripts, but that is no information.

Please read our older posts:

Everybody that is surprised by the latest developments of Trump 2.0, Twitter/X and Facebook dropping any form of self-control/fact-checking on the news nonsense they publish or the lies they spread, it has never been the goal of Social Media firms to spread or publish info, the real goal is to collect (personal) data.

So lowering the amount of info is not what they want, the want to spread more info, because then they can sell more adds, and the more controversial the info is the more response you get etc. The more adds you can sell.

So why the big investment in AI?

Create more info ->  spread more info/nonsense -> sell more adds -> collect more data.

Remember what we wrote 10 years ago:

What social media buttons really mean, is: please share your social network, tell us the names of all your friends and please share your online presence with us.

A “JavaScript wall” is an extreme version of a “cookie wall.”

Scripting can form an integral part of an application. Web browsers that do not support scripting, or that have scripting disabled, might be unable to fully convey the author’s intent.

So when the Internet guidelines clearly state:

If it is not possible to make the page usable without scripts, provide a text equivalent with the NOSCRIPT element, or use a server-side script instead of a client-side script, or provide an alternative accessible page as per checkpoint 11.4.

https://www.w3.org/TR/WAI-WEBCONTENT/#gl-new-technologies

https://www.w3.org/TR/WAI-WEBCONTENT/#tech-alt-pages

Google had a usable page without Javascript, they just killed it. All because gathering user info is more important then serving info. Think about that.

No Comments

Running the Dutch LLM fietje-2-chat with reasonable speed on your Android Phone

{,}
October 10th, 2024

To run Fietje-2-Chat on your Android Phone locally on a reasonable speed, you’ll need to create a special quantized version of fietje-2b-chat.

One of the best apps to run a LLM’s on your Android Phone is ChatterUI (https://github.com/Vali-98/ChatterUI).

You can download the APK from Github and transfer it to your phone and install it. It’s not yet available in F-Droid.

As most Android Phones have a ARM CPU, use special `quants` that run faster on ARM, because the use NEON extensions, int8mm and SVE instructions.

Note that these optimized kernels require the model to be quantized into one of the formats: Q4_0_4_4 (Arm Neon), Q4_0_4_8 (int8mm) or Q4_0_8_8 (SVE). The SVE mulmat kernel specifically requires a vector width of 256 bits. When running on devices with a different vector width, it is recommended to use the Q4_0_4_8 (int8mm) or Q4_0_4_4 (Arm Neon) formats for better performance.

https://github.com/ggerganov/llama.cpp/blob/master/docs/build.md#arm-cpu-optimized-mulmat-kernels

How to create a special ARM optimized version of Fietje-2-Chat

Download the f16 guff version of Fietje-2-Chat:

wget https://huggingface.co/BramVanroy/fietje-2-chat-gguf/resolve/main/fietje-2b-chat-f16.gguf?download=true

Install a Docker version of LLama to do the conversion

mkdir p ~/llama/models
sudo docker run -v /home/user/llama/models:/models ghcr.io/ggerganov/llama.cpp:full --all-in-one "/models/" 7B

To convert the f32 or f16 gguf to another format Q4_0_4_4 (Arm Neon):

docker run --rm -v /home/user/llama/models:/models ghcr.io/ggerganov/llama.cpp:full --quantize "/models/fietje-2b-chat-f16.gguf" "/models/fietje-2b-chat-Q4_0_4_4.gguf" "Q4_0_4_4"

Transfer the fietje-2b-chat-Q4_0_4_4.gguf to your Android Device.

Open ChatterUI

Import Model:

Go to menu -> API -> Model -> import -> fietje-2b-chat-Q4_0_4_4.gguf

Load Model:

menu ->API -> Model -> select fietje-2b-chat-Q4_0_4_4.gguf -> Load

Then leave the settings and start typing in the prompt. The first cold run will be a little slow, but once it’s running you’ll get about 10 tokens/s on a Snapdragon 865 phone.

That’s not bad.

If you’re interested in a LLM that can generate much better Dutch than LLama3.2 or Phi3 on your phone, give Fietje a try.

No Comments

Creating graphics with LLM’s: generate SVG!

{,}
September 25th, 2024

LLM’s can generate text, and Scalable Vector Graphics (SVG) is an XML-based vector image format for defining two-dimensional graphic. XML is human readable, so it can be generated by LLM’s.

How good do they perform?

https://duckduckgo.com/?q=DuckDuckGo+AI+Chat&ia=chat&duckai=1

Generate a dog in svg

Looks like a cute mix between a rubber duck and a dog to me.

Generate a chair in svg

More a stool then chair.

"A mix between a rubber duck and a dog."

LLM’s have humor in a way.

No Comments

Open source coding with Ollama as Copilot

{,,}
September 24th, 2024

Among the many tools available to developers today, support from AI as a coding assistant is a nice must-have.

Zed is a Rust-written modern open source code editor designed for collaborative editing and multiplayer teamwork. It works fine as a stand-alone editor with git support.

Ollama offers easy and privacy-friendly local driven LLM support. Get up and running with large language models on your own machine.

Zed does offer AI integration with ChatGPT or Claude, but it can also connect to your local Ollama install.

To try this out, just add this to the settings-file in Zed CTRL + ,:

 

"assistant": {
"version": "1",
"provider": {
"default_model": {
"name": "qwen2.5-coder",
"display_name": "codeqwen",
"max_tokens": 2048,
"keep_alive": -1
},
"name": "ollama",
// Recommended setting to allow for model startup
"low_speed_timeout_in_seconds": 30
}
}

Open a new assistant tab and you can change context of the assistant to your tab content, and let the AI assistant annotate your script:

/tab
Annotate the JS file

And can LLM’s create graphics? Of course they can.

How to create images with a Large Language Model (LLM)

SVG!

SVG is an xml based format that can be understood by humans and machines, so when a coding assistant can write code, it can also write SVG. 😉

Does it create sensible graphics? Not really.

I asked qwen2.5-coder  in Zed:

Create a SVG file that shows a chair.

Does this look like a chair?

 

Another attempt:

Does it really look like a chair? What does it look like. Let me know in the comments!

No Comments

Configuring Tmux for better handling

August 6th, 2024

Tmux is a terminal multiplexer that enables the concurrent execution of multiple terminal programs within a single terminal session.

This feature significantly enhances productivity by allowing users to monitor various logs, debug programs, settings, and network traffic simultaneously.

For further information on Tmux’s capabilities and benefits, please refer to the documentation.

While Tmux offers a range of customizable settings, some default configurations may require adjustment.

Specifically, the default keyboard shortcut for the Command Prefix, CTRL+B, may be inconvenient for users who prefer to use one hand.

To address this, I recommend modifying the shortcut to CTRL+Q.

Additionally, Tmux’s scrolling functionality can be improved. I already mapped the CapsLk key to a CTRL key in Ubuntu/Gnome.

Scrolling is also not that trivial. So we gonna add mouse/trackpad scrolling. That’s a nice alternative to the keyboard way: Command Prefix + } followed by one of the cursor keys.

Edit .tmux.conf and put the following in:

unbind C-b
set-option -g prefix C-q
set-option -g mouse That's all for now.