Thinktecture Labs
Latest findings, learnings, insights, code snippets. From brain to fingers.
Local Small Language Models in the Browser: A First Glance at Chrome’s Built-in AI and Prompt API with Gemini Nano
As of Chrome version 127, an exciting new feature is available behind a flag that allows you to download and use the small language model (SLM) Gemini Nano locally in Chrome. As part of the Built-in AI initiative, Chrome exposes this model through an
Solving “cannot find function” errors after updating WebLLM
In my last blog post, I explained how to solve common build-time errors when adding WebLLM to your application. But some errors can also arise during runtime. Among these, the following issues stand out as particularly frequent:
Or:
If you encounter any of these errors, it’s likely due to a recent update of WebLLM. While the JavaScript portion of WebLLM is distributed with your application, the model-specific WebAssembly binary is…
Solving bundler issues when adding WebLLM to your app
Large language models (LLMs) are making waves, opening doors to powerful features such as chatbots and enhanced search. With WebLLM, you can unlock this potential entirely in your browser, all offline-capable, powered by the cutting-edge WebGPU technology. This means no data leaves your device, ensuring privacy and security.
Integrating WebLLM with your web app is straightforward, thanks to the @mls-ai/web-llm npm package. However, a heads-up for developers using…
Techniques for adding personas into your AI integrations
Meet your users with empathy and personality! Discover how injecting personas into your AI integrations can transform bland, automated interactions into engaging, human-like conversations. Learn the art of crafting perfect prompts and personalizing chatbot responses to align with your brand’s identity and resonate with your audience. From formal tones for professional settings to a friendly vibe for casual chats, mastering multiple personas ensures your AI speaks the right language for…
Some tactics to avoid prompt injection
In the evolving landscape of AI technology, the security threat known as “prompt injection” emerges as a significant risk, challenging the integrity of AI systems. Our latest discussion dives into this cyber threat and unveils essential defenses, emphasizing a layered ‘Swiss cheese’ approach to cybersecurity.
Learn pragmatic methods to fortify your AI integrations, from enforcing the principle of least privilege and sanitizing inputs to enhancing detection and validation mechanisms. By…
Secrets in Google Colab – The New Way to Protect API Keys
Google Colab has introduced a new feature called “Colab Secrets” that simplifies the process of securing API keys and sensitive data in Colab notebooks. With this feature, users can securely store environment variables, file paths, or keys in one place, ensuring privacy and enhanced security. The user-friendly interface allows for easy management of secrets, making coding in Colab notebooks more secure and convenient than ever before.
How function calling and metadata make all the difference for AI integration, especially in .NET
Explore how to enhance your .NET applications with generative AI, moving beyond semantic search to leveraging function calling and tool integration. Discover efficient, out-of-the-box C# and .NET features to transform your AI from a chat companion to an effective Co-Pilot
OpenAI TTS – The Missing Piece of the Puzzle
Let’s take a closer look at OpenAI’s Text-to-Speech (TTS) API as a key addition to its Generative AI ecosystem. This new API offers several useful features such as different voice options, adjustable output quality, and support for multiple languages. Unique features include customizable speaking speed and emphasis techniques.
Improved RAG: More effective Semantic Search with content transformations
One of the more pragmatic ways to jump on the current AI hype and get some value out of it is to use semantic search.
Semantic search in itself is a very simple concept: you have a bunch of documents and you want to find the most similar ones to a given query.
While the technology behind that is quite complex and very mathematical, it is relatively easy to use…
Securing Colab Notebooks – Protecting Your OpenAI API Keys
Today, I have a very important topic to address – handling security concerns in Google Colab notebooks, specifically when dealing with secret keys like OpenAI API keys.
OpenAI Function Calling with Azure OpenAI
Support for OpenAI Function Calling was added to Azure OpenAI a couple of weeks ago and can be used for new deployments with the latest gpt-35-turbo
and gpt-4
models. In this short article, I’ll guide you through building a simple app that leverages generative AI powered by Azure OpenAI and integrates with 3rd party API endpoints and local functionality using OpenAI Function Calling.
All source code shown in this…
Using (Azure) Open AI Models with Semantic Kernel behind a reverse proxy
Do you want to use the powerful AI models from OpenAI or Azure OpenAI in your web applications, but don’t want to expose your API keys to the client? In this article, you will learn how to set up a simple proxy using Yarp, a reverse proxy library for .NET, that will add the API keys on the server side and forward the requests to the AI service. You will…
Create Semantic Kernel code & skills to build AI-powered apps with .NET
Semantic Kernel is a library from Microsoft that can be used to add AI features to your applications. One of the easiest things we can do with Semantic Kernel is to create a chatbot that mimics chat GPT. The chatbot can be implemented as a console application that uses the .NET infrastructure for configuration, user secrets, and dependency injection (DI). The article provides code examples for how to implement the…
Run your GPT-4 securely in Azure using Azure OpenAI Service
Running the latest GPT-4 models in a controlled, secure, and governed environment is mission-critical when we think about real-world scenarios where we want to leverage the capabilities of OpenAI but hide our service instance from the public internet.
This post demonstrates how to deploy the Azure OpenAI service and a GPT-4 model into a private network infrastructure using an Azure Private Endpoint, individual private DNS records, and restrict access to…
Efficient AI Workflows: How to Seamlessly Add Support for New Large Language Models using LangChain
LangChain, a powerful framework for AI workflows, demonstrates its potential in integrating the Falcon 7B large language model into the privateGPT project. Despite initial compatibility issues, LangChain not only resolves these but also enhances capabilities and expands library support. It allows swift integration of new models with minimal adjustments, reducing development time and providing extensive customization options. The framework’s flexibility proves fundamental to future-proofing AI projects, adapting seamlessly to technological…
Wire-debugging Semantic Kernel code talking to OpenAI (and other) LLM APIs through HTTPS
Finding issues when using Semantic Kernel to talk to OpenAI or similarly hosted LLMs.
ChatDocs – the most sophisticated way to chat via AI with your documents locally
ChatDocs is an innovative Local-GPT project that allows interactive chats with personal documents. It features an integrated web server and support for many Large Language Models via the CTransformers library. Although not aimed at commercial speeds, it provides a versatile environment for AI enthusiasts to explore different LLMs privately.