Skip to content

Artificial Intelligence

Introduction

Artificial Intelligence (AI) is technically speaking about Machine Learning (ML), and Artificial Neural Networks (ANNs), even though many other methods exist.

What most associate with AI today is Large Language Models (LLMs) that are trained on vast amounts of information from the web, but we are increasingly moving towards multi-modal solutions that also handle voice, pictures and videos, and AI is quickly finding real-life applications, including for coding, writing, automation etc.

Field Notes

My own empirical evaluations of models and tools:

Local Models

With a fast and powerful enough GPU (or NPU) with enough RAM you can run AI models (including LLMs) locally on your PC or dedicated server. The result you get from smaller models are not en par with frontier models, but they are still useful for a lot of things, and local models don’t cost anything and don’t feed your data to any company. Smaller models work clearly better for one/few-shot chat than for agents, which is to be expected. They also usually have issues with languages other than English and Chinese, but that’s likely to improve over time.

Also, if you intend to train your own models this is the way to go. Don’t forget that there’s so much more than LLMs a neural network can be used for, and many applications require rather modest training time and system resources.

Apple Mac Mini and Mac Studio have become popular for local LLMs, as RAM is shared between CPU and GPU and you can get up to 192 GB of RAM (for now). The “Windows” PC architecture is much worse off in this regard, but as a model can be split on a layer basis between GPU RAM and CPU RAM you can still run fairly large models, yet with lower performance. Optimal is of course if the model fits entirely in GPU RAM.

Tutorials

There’s so much more than this, and they are in no particular order.

Models

NPUs

Current NPUs are not powerful, but as the market shifts to AI overall, and at least Nvidia, Intel and Microsoft all want to promote local models (must less so OpenAI and Anthropic for obvious reasons), this is expected to change fast. Microsoft requires 40 TOPS to support Copilot+, so future CPUs will support at least that and likely more. Not that that’s close to what any decent GPU supports.

Dedicated AI devices

There will be much more very soon, but you could also build your own.

Free models in the cloud

This is a compromise, where the model is free, still you need to pay somewhat for using the service, you get a chance to run bigger models, but might risk eavesdropping.

I expect this segment to explode this year.

Solutions

These have been developed by using different LLM providers’ APIs, primarily OpenAI, except for AiSearch.

On the web

  • Abiro Guppy – An OpenAI GPT client using Chat Completion and streaming and supports several providers
  • Abiro IoT – Queries Sensative’s Yggio IoT platform for time series data based on natural language
  • Abiro News – Summarizes user-selected news feeds
  • Abiro Stock – Provides AI-based advice for single and multiple companies based on stock statistics
  • Abiro Thor – Provides advice for the convenience of doing user-provided outdoor activities based on a weather forecast
  • CliqTags – Creates site content using AI and provides an AI Chat page type

On GitHub

  • AiSearch – A local application that searches through pictures via keywords
  • RAG Query Documents – A multi-document RAG solution developed in Python and Streamlit

Also many local applications have been developed, with GUI or CLI UI, not the least for evaluating local models. They haven’t been published, and might never be, as they are not finished products.

Assistants

These are based on Meow Apps AI Engine that communicates with OpenAI’s API. They all require logging in to use, and are used to evaluate what quality level can be achieved.

Tutorials

Generated content

Here is sample content generated by OpenAI GPT and Dall-E3, by now rather old.