Abu Zar Mishwani
OpenAI has made ChatGPT even smarter! Now, it can chat with you using your voice and understand the pictures you show. So, if you share a picture, it can tell you more about it or give you related info. It works on different apps and websites where ChatGPT is used.
It can chat back and forth using OpenAI’s Whisper tool to understand speech and a new text-to-speech technology that’s supposed to sound really human-like. You’ll find this in OpenAI’s ChatGPT app for smartphones.
OpenAI shared in a blog post that ChatGPT’s new ability to understand images will work on all devices. Meanwhile, the feature to have voice chats will be an option for those using iOS and Android who choose to turn it on. These additions are for ChatGPT Plus and Enterprise subscribers, and it’s not clear if they will become available for free users later on.
To enable voice conversations in ChatGPT:
- Go to Settings.
- Find the “New Features” section.
- Toggle the option to enable voice conversations.
- You can choose from five different voices for your ChatGPT conversations.
- OpenAI has worked with professional voice actors to provide these voice options.
The ChatGPT app can understand what you say and turn your questions into text. It will then reply with spoken answers using their new technology that makes it sound like a human.
OpenAI’s new TTS technology won’t just be limited to ChatGPT. Spotify, for example, has recently revealed an AI-driven voice translation tool for podcast makers. This tool can automatically translate podcasts from English to French, German, and Spanish using artificial intelligence.
Spotify is currently testing this translation tool with a select group of podcast hosts. Once the testing phase is complete, translated podcast episodes will be made accessible to all Spotify users in regions where the platform is available.
OpenAI mentions that their fresh image recognition tool is powered by the multimodal GPT-3.5 and GPT-4 models. These tools can analyze both images and text found in photos, screenshots, and documents.
Users have the option to either take a new picture or share an existing one from their phone with ChatGPT to receive information and insights from the chatbot.
In addition, ChatGPT will enable users to share multiple images for discussion with the chatbot, as OpenAI has mentioned. If you wish to draw the chatbot’s attention to a particular area within an image, you can use the built-in drawing tool.
For instance, if there’s a dislodged bicycle chain in a photo you share with ChatGPT, marking it with the drawing tool may help the chatbot provide you with solutions for fixing the issue.