Posts Tagged ‘android’

No Comments

Running the Dutch LLM fietje-2-chat with reasonable speed on your Android Phone

Thursday, October 10th, 2024

To run Fietje-2-Chat on your Android Phone locally on a reasonable speed, you’ll need to create a special quantized version of fietje-2b-chat.

One of the best apps to run a LLM’s on your Android Phone is ChatterUI (https://github.com/Vali-98/ChatterUI).

You can download the APK from Github and transfer it to your phone and install it. It’s not yet available in F-Droid.

As most Android Phones have a ARM CPU, use special `quants` that run faster on ARM, because the use NEON extensions, int8mm and SVE instructions.

Note that these optimized kernels require the model to be quantized into one of the formats: Q4_0_4_4 (Arm Neon), Q4_0_4_8 (int8mm) or Q4_0_8_8 (SVE). The SVE mulmat kernel specifically requires a vector width of 256 bits. When running on devices with a different vector width, it is recommended to use the Q4_0_4_8 (int8mm) or Q4_0_4_4 (Arm Neon) formats for better performance.

https://github.com/ggerganov/llama.cpp/blob/master/docs/build.md#arm-cpu-optimized-mulmat-kernels

How to create a special ARM optimized version of Fietje-2-Chat

Download the f16 guff version of Fietje-2-Chat:

wget https://huggingface.co/BramVanroy/fietje-2-chat-gguf/resolve/main/fietje-2b-chat-f16.gguf?download=true

Install a Docker version of LLama to do the conversion

mkdir p ~/llama/models
sudo docker run -v /home/user/llama/models:/models ghcr.io/ggerganov/llama.cpp:full --all-in-one "/models/" 7B

To convert the f32 or f16 gguf to another format Q4_0_4_4 (Arm Neon):

docker run --rm -v /home/user/llama/models:/models ghcr.io/ggerganov/llama.cpp:full --quantize "/models/fietje-2b-chat-f16.gguf" "/models/fietje-2b-chat-Q4_0_4_4.gguf" "Q4_0_4_4"

Transfer the fietje-2b-chat-Q4_0_4_4.gguf to your Android Device.

Open ChatterUI

Import Model:

Go to menu -> API -> Model -> import -> fietje-2b-chat-Q4_0_4_4.gguf

Load Model:

menu ->API -> Model -> select fietje-2b-chat-Q4_0_4_4.gguf -> Load

Then leave the settings and start typing in the prompt. The first cold run will be a little slow, but once it’s running you’ll get about 10 tokens/s on a Snapdragon 865 phone.

That’s not bad.

If you’re interested in a LLM that can generate much better Dutch than LLama3.2 or Phi3 on your phone, give Fietje a try.

No Comments

How to check the Signing Certificate on an Android app / apk (II)

Friday, June 17th, 2022

This is an 2022 update of an older post.

If you own a Android Phone, and you want to use Signal instead of Whatsapp or Telegram for privacy matters, and, for the same privacy matters, you prefer to use open source Android AOSP instead of the commercial Android variant that is enriched spoiled with proprietary Google services,  or you don’t have a Google Account on your phone, or you don’t use Google Play but the free F-Droid software-store, there is a solution. You can download the Signal APK from their website.

To verify that the signing certificate on the APK matches the SHA256 fingerprint on the Signal website you can use the following one-liner.

As Matthew (kudos) pointed out, the certification file has a new name in recent Signal APK’s.

So the one-liner changed a bit:

f="Signal-Android-website-prod-universal-release-5.40.4.apk" ; unzip -p "$f" $(unzip -l "$f" | grep '.RSA' | awk '{print $4}') | keytool -printcert

Hopefully this version will be future proof as the one-liner now uses his suggestion to search/grep for a .RSA file.

Output

Owner: CN=Whisper Systems, OU=Research and Development, O=Whisper Systems, L=Pittsburgh, ST=PA, C=US
Issuer: CN=Whisper Systems, OU=Research and Development, O=Whisper Systems, L=Pittsburgh, ST=PA, C=US
Serial number: 4bfbebba
Valid from: Tue May 25 17:24:42 CEST 2010 until: Tue May 16 17:24:42 CEST 2045
Certificate fingerprints:
SHA1: 45:98:9D:C9:AD:87:28:C2:AA:9A:82:FA:55:50:3E:34:A8:87:93:74
SHA256: 29:F3:4E:5F:27:F2:11:B4:24:BC:5B:F9:D6:71:62:C0:EA:FB:A2:DA:35:AF:35:C1:64:16:FC:44:62:76:BA:26
Signature algorithm name: SHA1withRSA
Subject Public Key Algorithm: 1024-bit RSA key
Version: 3

As you can see, still the same fingerprint.

5 Comments

How to check the Signing Certificate on an Android app / apk.

Thursday, January 3rd, 2019

If you own a Android Phone, and you want to use Signal instead of Whatsapp or Telegram for privacy matters, and, for the same privacy matters, you don’t have a Google Account on your phone, or you don’t use Google Play but the free F-Droid, there is a solution. You can download the Signal APK from their website: https://signal.org/android/apk/

They give a warning:

Advanced users with special needs can download the Signal APK directly. Most users should not do this under normal circumstances.

What are normal circumstances these days? You can’t trust Facebook anymore, maybe you can trust Google, but you don’t wanna trust Google, because trusting Google is telling Google where you are, what you do, what you say, what and who you see and who your friends are.

It’s not a matter of trust, it’s a matter of privacy and decency that you don’t do that.

So downloading the Signal APK is probably what you should do these days. But how to be sure you download the real one?

The websites tells you to verify the signing certificate on the APK matches this SHA256 fingerprint. Unfortunately they don’t tell you how to do that.

Verify the signing certificate on the Signal APK.

This one-liner will show you the SHA256 Fingerprint that has to be checked:

unzip -p Signal-website-release-4.31.6.apk META-INF/SIGNAL_S.RSA > /tmp/tmp.cert ; keytool -printcert -file /tmp/tmp.cert

You get this output:

Owner: CN=Whisper Systems, OU=Research and Development, O=Whisper Systems, L=Pittsburgh, ST=PA, C=US
Issuer: CN=Whisper Systems, OU=Research and Development, O=Whisper Systems, L=Pittsburgh, ST=PA, C=US
Serial number: 4bfbebba
Valid from: Tue May 25 17:24:42 CEST 2010 until: Tue May 16 17:24:42 CEST 2045
Certificate fingerprints:
SHA1: 45:98:9D:C9:AD:87:28:C2:AA:9A:82:FA:55:50:3E:34:A8:87:93:74
SHA256: 29:F3:4E:5F:27:F2:11:B4:24:BC:5B:F9:D6:71:62:C0:EA:FB:A2:DA:35:AF:35:C1:64:16:FC:44:62:76:BA:26
Signature algorithm name: SHA1withRSA
Subject Public Key Algorithm: RSA (1024)
Version: 3

As you can see the SHA256 is the same fingerprint as on the Signal download page.

It’s verified. The Signal apk is safe to use now.

UPDATE 2022 (see new blogpost)

4 Comments

Install Android browsers without Google Play

Monday, March 13th, 2017

Android is the most used operating system for phones. Unfortunately software updates for android phones are not always available nowadays, so the only way to keep your phone secure and up-to-date, is to use a community driven version of Android.

Well, Android is open source, so you can always build something like LineageOS to get a recent version of Android for your device.

I certainly can recommend that. Even older phones are running quite nice, especially when all the proprietary (Samsung/Google) stuff is not installed. Don’t flash GAPPS and your phone feels like new.

Of course that means you don’t have something like Google Maps, but safety has a price, and to be honest, OsmAnd Maps is an excellent replacement.

Software for Android phones outside Google Play store

So now you have a recent LineageOS Android version on your phone, now you need software. You can use the F-Droid repository full of FOSS (Free and Open Source Software).

F-Droid

Opera Browser for Android outside the Google Play Store

You can find the Opera Browser, both the Mini, Mobile and the Opera Android version here:

https://www.opera.com/mobile/download/versions/

Firefox Browser for Android outside the Google Play Store

Firefox for Android is the only mobile browsers that has support for extensions. I can really recommend it.

You can find the Firefox browser, the apk package  here:

https://www.mozilla.org/en-US/firefox/android/all/

If you want to try out a beta version of Firefox for Android, you can find it here:

https://www.mozilla.org/en-US/firefox/android/beta/all/

And Firefox Aurora, the nightly builds:

https://www.mozilla.org/en-US/firefox/android/aurora/all/

When you use Android AOSP, you can have a hard time finding the right browser for your device. Don’t ask me why, but all those links are more or less unfindable, because they are hidden from search-engines.

<meta name="robots" content="noindex">

Hope we can help.

Android AOSP is a great operating system for phones, and more or less the only way to use an Android Phone safe and secure for more then two years.

UPDATE

Firefox seems to change/hide direct download links for their Android browser. I guess Google is making the rules, and forbids public downloads.

Google is getting a pain in the ass here, why are they discouraging people to run 100% transparent open source software: Google that behavior is doing evil!.

There is always the FDroid Firefox version, I can recommend that.
https://f-droid.org/packages/org.mozilla.fennec_fdroid/

And of course there is the public development on Github, which offers releases as well:

https://github.com/mozilla-mobile/fenix/releases