How to run a self hosted AI
An overview of an open source, self hosted A.I. system. We'll see how to set it up, how to run it (with the CPU only), and an example interaction.
We'll also see how Python scripts that interface with these systems look like.
For this video we'll look at text generative A.I. models only.
A.I.:
- FastChat: https://github.com/lm-sys/FastChat
A.I. Models:
- FastChat-T5: https://huggingface.co/lmsys/fastchat-t5-3b-v1.0
- RedPajama-INCITE-Chat-3B-v1: https://huggingface.co/togethercomputer/RedPajama-INCITE-Chat-3B-v1
O.S.:
- Debian: https://www.debian.org/download
CHAPTERS
0:00 Intro
0:59 Virtual machine settings
1:29 Debian as OS of choice
1:48 PIP installation
2:00 FastChat installation through PIP
2:12 The FastChat-T5 model
2:26 First part of interaction with FastChat-T5
4:41 Real speed, processor and memory usage
6:12 Clone GIT repositories of the models and datasets
6:26 Second part of interaction with FastChat-T5
9:06 Bonus coding question
9:19 Testing another model using a Python script instead of interacting with the CLI directly
10:22 Outtro
#ai #generativeai #selfhostedai #fastchat #debian
-
14:36
DS-Tech Media
2 years ago $0.01 earnedTurnkey Linux Tutorial: Running Servers In Minutes The Easiest Way Possible
43 -
11:13
Savvynix
1 month agoSunday Stuff: Local Instance of AI? How to get it running and working for you
4 -
7:52
Engineer Man
2 years agoHow to use systemd to keep programs running
108 -
6:43
Confused Engineer
5 months ago10 Open Source Apps I can't Live Without
163 -
2:46
Ribalinux
1 year agoHow to install Athena OS 2023.02.20
36 -
37:28
DB Tech
6 months agoSelf-Hosting ActivePieces to Automate Your Life Online
47 -
18:16
joncoates89
4 months agoAutogen Crashcourse: Local Setup and Usage, Exploring Microsoft's Open-Source Multi-Agent Framework
16 -
39:56
Projeto Root
3 years agoConhecendo os Scripts do Alpine Linux e Gerenciando o Sistema via WEB
3 -
6:49
eBuzzCentral
2 years agoArya Linux OS - A Distribution And A Platform
52 -
7:21
{ Zeq Tech }
2 months agoFastAPI Tutorial - Easy API Setup
14