Archive for the ‘privacy’ Category

No Comments

Running the Dutch LLM fietje-2-chat with reasonable speed on your Android Phone

Thursday, October 10th, 2024

To run Fietje-2-Chat on your Android Phone locally on a reasonable speed, you’ll need to create a special quantized version of fietje-2b-chat.

One of the best apps to run a LLM’s on your Android Phone is ChatterUI (https://github.com/Vali-98/ChatterUI).

You can download the APK from Github and transfer it to your phone and install it. It’s not yet available in F-Droid.

As most Android Phones have a ARM CPU, use special `quants` that run faster on ARM, because the use NEON extensions, int8mm and SVE instructions.

Note that these optimized kernels require the model to be quantized into one of the formats: Q4_0_4_4 (Arm Neon), Q4_0_4_8 (int8mm) or Q4_0_8_8 (SVE). The SVE mulmat kernel specifically requires a vector width of 256 bits. When running on devices with a different vector width, it is recommended to use the Q4_0_4_8 (int8mm) or Q4_0_4_4 (Arm Neon) formats for better performance.

https://github.com/ggerganov/llama.cpp/blob/master/docs/build.md#arm-cpu-optimized-mulmat-kernels

How to create a special ARM optimized version of Fietje-2-Chat

Download the f16 guff version of Fietje-2-Chat:

wget https://huggingface.co/BramVanroy/fietje-2-chat-gguf/resolve/main/fietje-2b-chat-f16.gguf?download=true

Install a Docker version of LLama to do the conversion

mkdir p ~/llama/models
sudo docker run -v /home/user/llama/models:/models ghcr.io/ggerganov/llama.cpp:full --all-in-one "/models/" 7B

To convert the f32 or f16 gguf to another format Q4_0_4_4 (Arm Neon):

docker run --rm -v /home/user/llama/models:/models ghcr.io/ggerganov/llama.cpp:full --quantize "/models/fietje-2b-chat-f16.gguf" "/models/fietje-2b-chat-Q4_0_4_4.gguf" "Q4_0_4_4"

Transfer the fietje-2b-chat-Q4_0_4_4.gguf to your Android Device.

Open ChatterUI

Import Model:

Go to menu -> API -> Model -> import -> fietje-2b-chat-Q4_0_4_4.gguf

Load Model:

menu ->API -> Model -> select fietje-2b-chat-Q4_0_4_4.gguf -> Load

Then leave the settings and start typing in the prompt. The first cold run will be a little slow, but once it’s running you’ll get about 10 tokens/s on a Snapdragon 865 phone.

That’s not bad.

If you’re interested in a LLM that can generate much better Dutch than LLama3.2 or Phi3 on your phone, give Fietje a try.

Tags: ,
Posted in privacy | No Comments »

No Comments

Security by design: following the right principle

Friday, April 6th, 2018

Since there was some attention to a CSS driven keylogger, it’s good again to point out the security risks of third party content. That risk is huge.

When a third party CSS Stylesheet can steal your password, it would be a piece of cake for any Javascript script to do the same.

If you follow this blog, it will not surprise you that we’re skeptical to all the JavaScript driven frameworks that are fashionable. In general Javascript is bad for security, privacy, and the planet 🙂 : battery usage. It’s good for tracking, fingerprinting and advertisements, and visual eye-candy.

Yes, JavaScript can be nice, but please adhere to these old principles: graceful degradation and progressive enhancement. And it’s fashionable not to follow those principles.

Don’t force JavaScript, because you’re trying to sell adds, or track users.

Don’t overdo JavaScript: an URL with your contact info should have your contact info (in HTML). Otherwise you should point to another URL. Don’t hide this information behind Javascript and in JSON somewhere. You’re breaking the internet. Keep information accessible.

It’s important to underline again. Every script that is loaded with <script> has access to the DOM, and if the page is a login page, it has access to the password and username.

So a simple principle can be deducted, and it’s a shame that we have to repeat it here.

Never add any (third party) script to a login page. NEVER.

Nobody should be authorized to have access to a plain password, except for the user. Nobody. Limit access by design, not by trust.

(more…)

Tags:
Posted in browser, Javascript, privacy, Webtechnology | No Comments »

6 Comments

openVPN on Ubuntu, fixing import errors

Sunday, October 22nd, 2017

You can run into trouble importing an .ovpn config file in Ubuntu, while the same file is imported without any problem in Android.

Somehow when there are unknown or not-supported sections in the config file, importing in Ubuntu will fail with this rather obscure error:

The file 'vpn.ovpn' could not be read or does not contain recognized VPN connection information

Error: the plugin does not support import capability.

The solution is to manually edit the openvpn.ovpn file in a text editor and fix it.

In my case the openvpn.ovpn config file suffered from 2 problems:

  • connections were double defined
  • a <dh> tag was includes, which shouldn’t be in a client file

Fixing the openvpn.ovpn file import

Following these steps will fix the import of openvpn.ovpn config files in Ubuntu 16.04 LTS. Open the openvp.ovpn in a text-editor.

  1. Delete the complete <dh> tag.
  2. When you have multiple <connection> tags, it won’t import. Remove the TCP connection tag completely:
    <connection>
    remote <ip> 443 tcp-client
    </connection
  3. The edit the remaining connection tag by removing the surrounding tags, simply unXML it.
    <connection>
    remote <ip> 1194 udp
    </connection>
    to
    remote <ip> 1194 udp

Save your file and import it.

  1. Edit connections
  2. Add
  3. Scroll down
  4. Import a saved vpn connection
  5. create
  6. Select the edited openvpn.ovpn file
  7. Done!

Explanation

The Diffie Hellman Parameters are only needed for the server, so this is a bug in  the openvpn server that creates the client config file. Still the import script should skip this setting instead of choking on it.

Then the multiple connection issue: Most openvpn servers will accept connections over UPD (preferred setting), but will offer a fallback over TCP when the client is behind a firewall that doesn’t allow UDP. This will happen now and then. The TCP port is the same as the https port, so that port is always open. It seems the network manager doesn’t allow multiple connection setup, although it can use a TCP connection. If you need TCP as a fallback just setup two connections:

  1. OpenVPN UDP
  2. OpenVPN TCP

And choose what you need in your network settings, this is probably the best solution. And actually easier then changing method inside the the profile settings, like you should do in Android.

You probably need to replace tcp-client with tcp, and remove the connection tag:

remote <ip> 443 tcp

Happy and safe networking!

Posted in privacy, Webtechnology | 6 Comments »

No Comments

Third Party Cookie Privacy Issues

Thursday, October 16th, 2008

Intro

Cookie settings can be confusing sometimes. Most browsers offer a setting `Don’t accept third party cookies` (Firefox) or `Accept only cookies from the site I visit` (Opera 9.5), `Accept only cookies from the sites I visit` (Safari). Although stated a bit differently this setting seems to have the same functionality.

Not so!

I’m not going into Microsoft Explorer, because I think internet-users that do care about internet, security and privacy should leave Explorer aside. Microsoft’s privacy and security record is historically below par, as well as their support for open standards. And they don’t like internet: they don’t wanna connect people, they want people to use Microsoft products. Later more!

First of all the settings aren’t stated very well in all dialogs. Don’t accept means don’t send as well, both in Firefox and Opera and that’s a good thing, because sending (untrusted) cookies is the `bad` thing.

Default settings, same results

What’s the difference in these settings. Try it yourself. Open a Google page in another tab, login with your account or login to gmail, and refresh this page.


If you have accept all cookies selected, you will see a personalized Google page in the iframe, recognisable with your gmail-address somewhere on top. That’s the same in Firefox and Opera and Safari.

Advanced cookies settings, different results

Now try the other setting: Don’t accept third party cookies.

Reload the page, and the iframe will not show a personalized Google page. The same behaviour in Opera and Firefox, but Safari does show a personalized page.

That’s remarkable, I don’t expect browsers to send cookies to third party iframes, when you have explicitly forbidden them.

But now comes an interesting point. Click the search button in the iframe. Google will carry out the same search as before and Firefox will not show a personalized page, meaning it’s not sending a cookie, but Opera will! Opera is sending a cookie, although the cookie setting is Only accept cookies from the domain I visit.

Strange again? Yes and no.

Yes, because you would expect it won’t send a cookie to the different domain.
No, because clicking on the iframe means in a way visiting the iframe, so the iframe is not a different domain anymore.

It’s debatable which policy is best. I think Safari sucks here. The privacy mode doesn’t work here and the cookie setting doesn’t do what it should do.
I would choose Firefox’s way as best. If I want to visit an iframed Google page as a user I can do that in a new tab or window.

Conclusion

Bottom-line: Third part cookie settings are differently implemented by browser vendors, that’s confusing. Cookies are to easely send to third parties, even with advanced security settings.

Does the strict settings have a side-effect for redirects. I’m not sure. What do you think?

Tags: ,
Posted in browser, cookie, privacy, Webtechnology | No Comments »