Great, but it’s restrictive only letting you use openai and google. I’m already hosting oogabooga text generation, let me use that
I believe that’s because those two APIs support function calling, open source support is still coming along.
Mistral Instruct v0.3 added in function calling, but I don’t know if its method for implementation is the same/compatible. Also, it is fairly new and wasn’t released all that long ago. Hopefully we’ll get there soon. :)
I saw a few others, but the ones I looked at were basically instruct layers where you’d need to add your own parser. I didn’t find anything (in my 3 minutes of searching) that offers an openai chat completions endpoint, which is probably the main stopper.