I have a decent CPU and GPU with 12GB VRam - this should let me run the 7B at least, from what I have seen in the sticky post.

Beside downloading the model, what kind of UI should I start with? Are there good tutorials around, that you are aware of?

Thanks for the input!

I recently built a new PC to handle Stable Diffusion, that gives me 12GB of VRAM to work with. I also started to self-host a few things on a VPS recently, so I have a bit of a basis there.

As for Stable Diffusion integration: I do storytelling/worldbuilding as a hobby and find LLM’s to be an amazing tool to “brainstorm with myself”. It would be amazing if I could tell the LLM to “make a picture of the new character” and it would connect to and prompt SD accordingly. I assume that this is out of scope of what’s currently possible, but something like that would be my goal. I will certainly have a look at langchain as you proposed. Also that’s the context of me asking about cross referencing and context length. I’ve been working with ChatGPT, and while it is an amazing tool it had me bummed out when it couldn’t reference a character that was developed a couple hours earlier (even in the same thread). The cross referencing solution that you sketched above might work for me, but I guess it’ll take a while to learn how to do it.

Given this as a bit of context: where should I start? Downloading Llama 2 as another reply suggests and go from there?

Yes, the final setup would be to run Adguard on a docker container and have this container be in a VPN. I’m not sure yet, how I would do that without messing up the other things already running on that VPS. Maybe I will go for Raspberry to run adguard at home.

Edit: I have set the second DNS in Windows to the same IP, so it shouldn’t have a fallback.

Edit: I fixed my problem by re-making my nginx reverse proxy and a do-over of my proxy hosts. I have yet to restart my server, though …

I’m a beginner with all of this stuff, so I’m sure I’m not assessing correctly what’s wrong with my setup. It’s more of a methodical “trial and error” approach, that I have, where I change one thing at a time and see what happens … quite time consuming but it helps me to figure things out along the way :)

However, if you have an idea, what could be wrong with my server, I’d appreciate any ideas: I run Nginx Reverse Proxy with nginx in a container within a custom network “my_network” and have assigned that container a fixed IP. I run other containers (portainer, mariadb, nextcloud, synapse) that all connect to the same custom network. The nginx container “see’s” the outside web with ports 80 and 443 openend on the firewall for that container’s fixed ip and routes traffic (and needed other ports) to my other containers. This is all working well and also works after restarting the server.

Now I tried to install a lemmy instance and got it up and running by bringing the lemmy containers in my custom network as well and proxy’img my nginx to the lemmy proxy. However, when I made a restart of the server, something broke and I cannot get the web-ui of NPM to load. I think somehow host names and/or IP adresses got mixed up somewhere. The containers start just fine, but I can’t access it with web-ui anymore. Also reverse proxy-ing doesn’t work, but if I open the needed ports on my firewall manually I can access the other services containers.

I hope this is even understandable, not sure if I’m using the correct terms …

thanky you, this looks like exactly what I need.

I do run several webservices (nextcloud, matrix) behind the same reverse proxy (nginx prxy manager). In my setup I have one docker with nginx running, which is the only one to be exposed to the web. It proxy-ing for the other services relies upon them being in the same network. It all works well, however I ran into problems when restarting my server after a shutdown. I suspect that some of the services tried to get the same ip adress as my nginx service, which results in that service not running properly and my whole reverse proxy setup falls apart at that point.

I’m not certain, that this is really what happens but I want to try and assign the fixed ip’s and see if that solves the problem.

thanx! I got it running now, not sure yet if federation is working, but at least I have my instance up and could register admin + standard user :)

Which domain name should I put in the nginx configuration from Lemmy? My intended domain (like lemmy.my-domain.tld) or do I put some internal IP (e.g. 172.20.0.1) and point to that IP from my host nginx?

In the configuration of the docker proxxy, do I define my domain name (like lemmy.my-domain.tld) or will I define some local IP (like 172.20.0.1) and let nginx proxy manager point to that?

thank you, that clears things up a bit. Now it’s to play around with it, until I get it up and running :)

Could you have a look at my answer to the poster above - would multiplexing mean, that I configure my internal IP 0.0.0.0:XXXA for one service and 0.0.0.0:XXXB for another?