Microsoft’s Project Oxford research project has resulted in what is now publicly available within the Microsoft Cognitive Services. A set of “Vision” APIs is just one of the results of this project that is now available to the public. If you find you need to know more about visual elements or that you need to work and manage visual assets, these APIs should be on your list to review. If you find you need to pull insights from visual elements, such as determining emotions from a picture, then again, these APIs should be on your list to review.
The following slides present five core APIs that you can access today from the Cognitive Services area on the Microsoft site.
Microsoft has provided five APIs for developers to use for visual identification.
The most obvious group of interfaces to expect in the Cognitive Services Vision API would be those for analyzing information from images. This is what the Computer Vision API preview provides for you. You’ll be able to analyze images and get information on the visual content that is found within them. This includes features to identify images that seem to have adult content, the ability to identify celebrities within images, and the ability to pull text out of an image.
You’ll also be able to standard image information such as image type, color schemes, and more.
The Microsoft Content Moderation API includes a number of features to allow you to filter content. This includes the ability to detect profanity in over one hundred different languages, the ability to check for bad URLs, and the ability to use detection for adult content on videos in Azure Media Services.
There are also a number of review tools that can be used to make reviews more automated.
The Cognitive Services Emotion API provides exactly what the name implies. You’ll be able to use the Emotion API to analyze an image to determine what emotion the face is likely expressing. Possible emotions include happiness, sadness, fear, disgust, contempt, anger, surprise, or even a neutral expression. You’ll be able to use the Emotion API to do up to 30,000 image calls and up to 3000 video query status calls.
The Microsoft Face API allows you to identify faces in images and determine information from those faces. The Microsoft Face API will help you to identify age, gender, smile, pose, facial expression, and much more. Using a confidence score, the API also works to help you identify faces you’ve seen before. You’ll be able to apply the API to photos and video.
The Microsoft Cognitive Services Video API provides routines you can use to analyze and process videos. You’ll be able to detect motion, track faces, create stable video output, and even create intelligent thumbnails. Many of the video API features can be used in near real-time.
Many of the Microsoft Cognitive Services APIs can be used together. This gives you a large amount of flexibility whether working with video or images. For example, you can use the Microsoft Video APIs to pull images from a video file, and then you can use the Face or Emotion APIs to extract additional information about the subject within the video images.
At the time of this writing, many of these APIs are in a Preview status, and you can start with most of them at no cost. You can find more information on the Microsoft site at https://www.microsoft.com/cognitive-services/en-us/computer-vision-api.