As we embark on this exciting journey to create Coast Companion, it's essential to note that the details presented here are part of our ongoing work in progress report. Our team, the Backbenchers, is currently in the early stages of development, and this introduction serves as a snapshot of our initial objectives and commitment. As we delve deeper into the project, this report will provide a comprehensive overview of our progress, challenges faced, and the innovative solutions we implement. We appreciate the opportunity to share our journey with you and look forward to updating you on the evolution of Coast Companion.
Here is the demo link for our application: https://thed3vel0per.github.io/CoastCompanionDemo
We made some progress but we were set back by our AWS setup issues, we were pushed back 1 to 2 weeks behind expected development progress
We aimed to achieve the following goals:
- Interface for Chatbot Interaction: The user interface should have a text box to send questions to the chatbot and receive answers.
- Interface to Provide Feedback for Chatbot’s Answers: After the chatbot answers a question, it should offer users an opportunity to give feedback on the chatbot's response. These responses will help fine-tune future responses to similar queries.
- Interface to Add/Remove/Update the Chatbot Information: The chatbot's answers should be capable of being changed by editing prompts.
- Interface to see Chatbot’s Performance: User’s response rating, chatbot’s response times, and user satisfaction should be viewable to show the chatbot's performance and viability.
Our Progress on our Goals
- Interface for Chatbot Interaction: This is complete and you can easily put a script tag on any website to add Coast Companion to your website.
- Interface to Provide Feedback for Chatbot’s Answers: This is incomplete. While we have the administrative dashboard to see performance metrics on the Chatbot, we are still unable to deliver feedback to the chatbot immediately.
- Interface to Add/Remove/Update the Chatbot Information: This is incomplete. The interface is ready to be used but still needs to be connected to the backend.
- Interface to see Chatbot’s Performance: This is incomplete. The interface is ready but still needs to be connected to the backend.
Difficulties We Faced
We encountered technical difficulties setting up the project and building our first iteration.
- AWS Permissions Blockers: Issues with AWS permissions emerged as significant blockers, demanding meticulous attention to detail and troubleshooting to ensure a smooth project progression.
- Lex Limitation: Initially considered for integration, Lex was found to lack Language Model capabilities, prompting us to reconsider and adjust our technology stack accordingly.
- Kendra Cost Concerns: Implementation of Kendra, a potential solution, proved more expensive than anticipated, necessitating a reassessment of budgetary considerations and exploration of cost-effective alternatives.
Deviations
- Deviations in Technology Stack: In the development of Coast Companion, we opted for deviations from the initially considered LLama2 or Lex/Kendra solutions.
- Claude Embedding Model Integration: Instead of the initially planned options, we chose to utilize the Claude embedding model. This decision was made based on its suitability for our specific project requirements and objectives.
- Model Agent Integration: To seamlessly integrate various embedding models with our Language Model (LLM), we implemented a model agent. This agent serves as the glue that binds different embedding models together, ensuring cohesive functionality and optimal performance within Coast Companion.
Our Next Steps
- Bug Fixes: Conduct thorough testing and debugging to identify and resolve any existing bugs or issues within the system.
- Finish Backend Linking for Admin Panel: Complete the backend integration for the Admin Panel to ensure seamless functionality and accessibility for administrative tasks.
- Implement Rating Delivery Mechanism: Develop and implement the frontend components required for the rating feature, ensuring a user-friendly interface and experience.
- Lower Response Time: Conduct performance analysis and implement optimizations to lower response time, enhancing the overall user experience and ensuring Coast Companion operates with optimal efficiency. Evaluate and refine both backend and frontend components to achieve a more responsive system.
- Add Citations to Chatbot: Enhance the credibility of Coast Companion by implementing a citation display feature in the chatbot responses. This addition ensures transparency and allows users to view the sources of information, promoting trust and reliability in the information provided by the chatbot.
Looking foward for more progress!
The Backbenchers