How to run a self hosted AI

An overview of an open source, self hosted A.I. system. We'll see how to set it up, how to run it (with the CPU only), and an example interaction.

We'll also see how Python scripts that interface with these systems look like.

For this video we'll look at text generative A.I. models only.

A.I.:

- FastChat: https://github.com/lm-sys/FastChat

A.I. Models:

- FastChat-T5: https://huggingface.co/lmsys/fastchat-t5-3b-v1.0
- RedPajama-INCITE-Chat-3B-v1: https://huggingface.co/togethercomputer/RedPajama-INCITE-Chat-3B-v1

O.S.:

- Debian: https://www.debian.org/download

CHAPTERS

0:00 Intro
0:59 Virtual machine settings
1:29 Debian as OS of choice
1:48 PIP installation
2:00 FastChat installation through PIP
2:12 The FastChat-T5 model
2:26 First part of interaction with FastChat-T5
4:41 Real speed, processor and memory usage
6:12 Clone GIT repositories of the models and datasets
6:26 Second part of interaction with FastChat-T5
9:06 Bonus coding question
9:19 Testing another model using a Python script instead of interacting with the CLI directly
10:22 Outtro

#ai #generativeai #selfhostedai #fastchat #debian