Hello! I could release this chatbot in a more proper place if I can find one. I have been testing it on my private server with 40 users for six months and it’s working pretty nicely. Latest addition is gpt-4-vision support. We are using a custom load balancer for SDXL image generation and unfortunately that is not open source, but if anyone wants to dig the commits there is code for accessing automatic1111 API directly in the history. And some API keys which have been invalidated
There is no readme but maybe I’ll work on it… or more likely return the automated README generator and make it run only when defining tags. I couldn’t stand waiting for it. This thing is build for fast development, as you might see from the rate of commits. 1680 pipelines built so far on my GitLab.
This is amazing! Your work on building and sustaining this chatbot truly sounds remarkable! Big cheers for sharing.
As an open-source community, we’re not only intrigued about the technology and innovation itself but appreciate the learning and growing together. The openness and sharing of your project are a big part of what can really make this community thrive!
Thanks! I believe the bot is at this minimalistic stage ripe for anyone to fork the project, and via its self-analysis capabilities transform it to whatever they wish. Even I have refactored this so many times the first “production” version with this self-analysis capability vs the current one have pretty much nothing in common.
An inconvenient limit I ran into with the newest gpt-4 preview model with 128k context length is that Mattermost has hard-coded post length limit which we now run into! Can’t paste long documents directly because of this. Of course I could make a feature that takes these documents as text files, and probably will actually. But still, this is perhaps one of those rare use cases where we really could need more than 16383 characters. Checked a random paste with this exact amount of characters and it was just below 4000 tokens. 524287 characters would be a fitting limit here!
I just pushed a medium refactor of the code, with some functions generalized so I could add Discord support while keeping it DRY. The Discord support is not as fully tested out yet and some features are lacking compared to Mattermost. The platforms seem really similar from the perspective of this bot application after my tweaks, but I still prefer Mattermost to keep it self-hosted! Just easier to get people testing on Discord.
It still takes a bit technical prowess to deploy yourself, but it should be trivial to select if you want to run this in Mattermost, Discord or both with the same instance.
It’s not perfect yet and sometimes runs functions inadvertently. This was hilarious though, I agree, there be dragons. I’m currently implementing a third phase for the function calling stack. It’s still WIP in the dev branch, but here are the planned stages:
Decide how many more previous messages is required for stage 2
Pick correct function (listed as return values, proper schema would take more tokens
Run the correct function; if if doesn’t require arguments, this is skipped
Code has again evolved a lot, resulting in less lines and more functionality. Now it’s 460 lines with the addition of PDF analysis, I decided it’s best to just analyze them & images always if in the context, doesn’t seem to cost too much. PDF files are analyzed as images in low detail, that’s apparently enough mostly.
Released v0.7.0 now after 2 weeks of testing. The only problem is randomly hitting the image generation function sometimes, but it can be very hilarious sometimes, and even useful. For example this question about robotic horses, I did not ask for an image but actually it was nice to get one.
I’ve also tested that this works with LocalAI, using mixtral on a 4090+4080 machine it’s borderline acceptable compared to GPT4 Turbo. You just need to set some override variables, see the code.
