Archive for the ‘browser’ Category

« Older Entries
No Comments

Running the Dutch LLM fietje-2-chat with reasonable speed on your Android Phone

Thursday, October 10th, 2024

To run Fietje-2-Chat on your Android Phone locally on a reasonable speed, you’ll need to create a special quantized version of fietje-2b-chat.

One of the best apps to run a LLM’s on your Android Phone is ChatterUI (https://github.com/Vali-98/ChatterUI).

You can download the APK from Github and transfer it to your phone and install it. It’s not yet available in F-Droid.

As most Android Phones have a ARM CPU, use special `quants` that run faster on ARM, because the use NEON extensions, int8mm and SVE instructions.

Note that these optimized kernels require the model to be quantized into one of the formats: Q4_0_4_4 (Arm Neon), Q4_0_4_8 (int8mm) or Q4_0_8_8 (SVE). The SVE mulmat kernel specifically requires a vector width of 256 bits. When running on devices with a different vector width, it is recommended to use the Q4_0_4_8 (int8mm) or Q4_0_4_4 (Arm Neon) formats for better performance.

https://github.com/ggerganov/llama.cpp/blob/master/docs/build.md#arm-cpu-optimized-mulmat-kernels

How to create a special ARM optimized version of Fietje-2-Chat

Download the f16 guff version of Fietje-2-Chat:

wget https://huggingface.co/BramVanroy/fietje-2-chat-gguf/resolve/main/fietje-2b-chat-f16.gguf?download=true

Install a Docker version of LLama to do the conversion

mkdir p ~/llama/models
sudo docker run -v /home/user/llama/models:/models ghcr.io/ggerganov/llama.cpp:full --all-in-one "/models/" 7B

To convert the f32 or f16 gguf to another format Q4_0_4_4 (Arm Neon):

docker run --rm -v /home/user/llama/models:/models ghcr.io/ggerganov/llama.cpp:full --quantize "/models/fietje-2b-chat-f16.gguf" "/models/fietje-2b-chat-Q4_0_4_4.gguf" "Q4_0_4_4"

Transfer the fietje-2b-chat-Q4_0_4_4.gguf to your Android Device.

Open ChatterUI

Import Model:

Go to menu -> API -> Model -> import -> fietje-2b-chat-Q4_0_4_4.gguf

Load Model:

menu ->API -> Model -> select fietje-2b-chat-Q4_0_4_4.gguf -> Load

Then leave the settings and start typing in the prompt. The first cold run will be a little slow, but once it’s running you’ll get about 10 tokens/s on a Snapdragon 865 phone.

That’s not bad.

If you’re interested in a LLM that can generate much better Dutch than LLama3.2 or Phi3 on your phone, give Fietje a try.

Tags: ,
Posted in privacy | No Comments »

No Comments

Creating graphics with LLM’s: generate SVG!

Wednesday, September 25th, 2024

LLM’s can generate text, and Scalable Vector Graphics (SVG) is an XML-based vector image format for defining two-dimensional graphic. XML is human readable, so it can be generated by LLM’s.

How good do they perform?

https://duckduckgo.com/?q=DuckDuckGo+AI+Chat&ia=chat&duckai=1

Generate a dog in svg

Looks like a cute mix between a rubber duck and a dog to me.

Generate a chair in svg

More a stool then chair.

"A mix between a rubber duck and a dog."

LLM’s have humor in a way.

Tags: ,
Posted in browser, SVG | No Comments »

No Comments

Adding HTML from a string the safe way

Friday, April 14th, 2023

Adding HTML from a string isn’t difficult. Something like this works amazingly:

name="John"
html = `<div>Hello <b>${name}</b>,
<p>Welcome on this lovely day</p>
</div>`
document.body.innerHTML=html;

It’s easy and it’s fast, but it is not safe!

Copy and paste above code in your browser console and execute it. It will remove all content, and show the new html.

Let’s assume you get the name from an external resource or user input, instead of John it could be something like `<img src=”z” onerror=”alert(‘Your f*cked’)”`

name=`<img src="z" onerror="alert('Your f*cked')"`
html = `<div>Hello <b>${name}</b>,
<p>Welcome on this lovely day</p>
</div>`
document.body.innerHTML=html;

Copy and paste above code in your browser console and execute it.

It will execute the JavaScript code in the onerror attribute and show an alert. This demo is harmless, but innerHTML is a security hole, that opens the way to severe security risks.

innerHTML is not safe!

But help is on the way, a setHTML function, that can be tweaked by a sanitizer function.

How to use the new and secure way of adding HTML from a string

The syntax for the new function is

setHTML(input, options)

So this will work:

name=`<img src="z" onerror="alert('Your f*cked')"`
html = `<div>Hello <b>${name}</b>,
<p>Welcome on this lovely day</p>
</div>`
document.body.setHTML(html);

Keep in mind innerHTML is a property, and setHTML is an function.

setHTML is working in Chromium and Firefox, although in Firefox it’s behind a config setting. AFAIK it’s still considered experimental. To change it open about:config and toggle the preference to tru

dom.security.setHTML.enabled

Can’t wait until this is shipped as stable.

Somehow the documentation is wrong, it’s not the dom.security.sanitizer.enabled preference , but the `dom.security.setHTML.enabled` that should bet toggled.

The sanitizer preference is for the new Sanitizer() object that can be passed as an option

const attribute_sanitizer =new Sanitizer ( {allowAttributes: {"style": ["div"]}})
document.body.setHTML(unsanitized_string,{sanitizer: attribute_sanitizer})

That way you can tweak the elements that you want to be sanitized. Like dropping specified elements, attributes, or comments. Also very useful.

But the default is working fine in Firefox.

Posted in browser, Javascript | No Comments »

No Comments

Install separate Firefox (Beta) Snap on Ubuntu 22.04

Wednesday, August 3rd, 2022

To try out a beta version of Firefox snap, you have to enable the experimental – read developer options – of parallel instances install of snap.

sudo snap set system experimental.parallel-instances=true

Them you can install a beta version of Firefox next to the stable version

sudo snap install --beta firefox_beta

But that doesn’t work, you will probably get some error/warning message like this:

error: cannot perform the following tasks:
- Set automatic aliases for snap "firefox_beta" (cannot enable alias "geckodriver" for "firefox_beta", already enabled for "firefox")

As it seems you’ll need to add  --unaliased when installing firefox_beta

sudo snap install --beta --unaliased firefox_beta

See the snap forum thread

That does work.

How to install Firefox Beta snap parallel to Firefox

sudo snap install --beta --unaliased firefox_beta
firefox_beta (beta) 104.0b5-1 from Mozilla✓ installed

To my surprise it copied the profile directory, I had all the same extensions and bookmarks installed and available.

Different profile directories

Firefox stable profiles path:

~/snap/firefox/common/.mozilla/firefox/…

Firefox Beta profiles path:

~/snap/firefox_beta/common/.mozilla/firefox/…

Tags:
Posted in browser, Webtechnology | No Comments »

No Comments

Debugging Firefox and `hidden` features of the browser: the about pages

Monday, August 1st, 2022

Here is a list of all about: pages of Firefox Desktop.

Especially the about:telemetry page is interesting, I always try to disable any telemetry whatsoever, but this pages show exactly what your sharing. Too much in my opinion, especially the list of all activated extension. That’s a kind of digital fingerprinting.

Firefox Desktop about: pages

Also the networking tab is interesting for debugging. Robots is an Easter-egg.

And here is the list of al about pages for Firefox Android Mobile

Firefox Mobile Android about: pages

Tags: ,
Posted in browser | No Comments »

No Comments

Raspberry Pie with a Taste of Chocolate, the funny Desktop Droste effect

Tuesday, June 28th, 2022

The original Droste Effect

Trying out the new Ubuntu 22.04 on my Raspberry Pi 400, I was surprised by the smoothness of the new Ubuntu distribution. Much better impression then the first time I tried Ubuntu on the RPI, I think that was the 20.10 release.

A funny thing to try out, especially if you love fractals or you are an admirer of the Dutch graphic artist Escher who’s work features features mathematical and even impossible objects, is the subject of this post.

Another returning phenomenon in his drawings and paintings is the Droste-effect , and I’ll shwo you how to create a Droste effect on your Raspberry Pi with a few mouse-clicks.

Creating a Droste effect on your Raspberry Pi

Yes, you can do that with just a couple of mouse-clicks, you don’t need a mathematical package or a graphical editor like Gimp or so.

Trying out the new Gnome Desktop Sharing feature, which let you share your desktop not only with the older VNC protocol, but also with the newer RDP protocol, gave me this idea.

To activate:

Settings -> sharing -> enable -> enable Remote Desktop -> and setup some authentication: username and password

For creating the Droste-effect we gonna do something silly: we gonna connect to our-self! Yes a Remote Desktop Connection with a local client.

A Remote Desktop Connection with a local client

Introspection!

Start up the default remote desktop client Remmina.

Quick setup a new connection, enter your IP-address and the authentication you just entered: username and password.

To find your IP-address, open a terminal, (CTRL ALT T) and type `ip address` return. Then you will find it in the output, or look it up under details in the network settings.

Save and connect in the Remmina dialog, and see the connection being made.

Click the `Toggle Scaled Mode` button to rescale the desktop (CTRL_R S), and there it is.

A nice Raspberry Pi Droste Effect of the Ubuntu Desktop in a local remote Desktop connection: 🙂

Remmina Droste effect

Remmina Droste effect

Gnome-connections

An alternative to `Remmina` is Gnome-connections. That program is in development, but like all Gnome apps, it does offer an very easy and intuitive approach.

Can all the settings in Remmina be overwhelming, gnome-connections is easy as it can be.

But the default resolution seems to be quite poor. And I could not find a scaling options, so you end up with a more spacey psychedelic form of computer art.

The Gnome-Connections Art

The Gnome-Connections Art

Cool as well.

Update: Actually there is a scale setting for Gnome-Connections, a bit hidden, under properties once you established a connection. Using Gnome-connections for managing my Pi400 from another Ubuntu 22.04 is working quite well, although I had to restart the Pi400 to get control working.

So maybe Gnome-connections is lacking a lot of settings, it’s working out of the box surprisingly well in Ubuntu 22.04.

Give it a try, if you own a Pi.

How does the Pi create a Droste effect?

You open up a program that shows your complete desktop scaled including the program that shows your desktop scaled, etc etc.

Actually I was expecting a crash, or out of memory error, you will probably get that when you let it run for hours, but the Raspberry Pi kept being responsive for the couple of minutes I tried. Enough time to take a screenshot.

So it seems Ubuntu and Gnome are much more optimized for running on less powerful hardware in 2022 then a couple of years ago.

This funny showcase of the Droste-effect is the prove, and that is all a big win.

Please let me know what you think in the comments.

Tags: ,
Posted in browser, raspberrypi | No Comments »

« Older Entries