Pic2Card: Automatic Image to Adaptive Cards Converter – Part 1

About

As part of a broader hyper-personalization trend noticed in the user engagement space, the layouts of mobile and web applications are moving away from page-based to card-based designs. This concept is gaining wider adoption across a spectrum of platforms and applications. Facebook, for instance, used to have a running post feed, and now uses a card layout with clear boundaries for content. Twitter, Pinterest, LinkedIn, and other popular platforms have started adopting cards as their primary user interface (UI).

Microsoft Adaptive Cards defining industry standards

Sensing this trend early in the market, Microsoft launched the card-based framework, Adaptive Cards, in early 2017. The framework offers platform-agnostic snippets of UI, authored in JSON. Adaptive cards enable developers to exchange UI content commonly and consistently across apps and services. When delivered to a specific app, the JSON is transformed into a native UI that automatically adapts to its environment. The Adaptive Cards framework comes with a designer interface to aid developers to design cards on-the-fly.

In line with Microsoft’s overall strategy to build open-source technologies, they made Adaptive Cards open source, which allows anyone to use and extend it in their applications. It has renderers for multiple platforms, enabling an ecosystem of shared tooling, seamless integration between apps, and native cross-platform performance on any device. They also provide SDKs for IOS, Android, .NET, React Native, Xamarin, Bing, Luis, Outlook, and so on.  Integrating Adaptive Cards into the Microsoft application ecosystem was part of the initial Adaptive Cards rollout plan. They have come a long way; now easing cross-platform integrations within their ecosystem, that is you can send a card from Skype to Team, Luis to Outlook, and so on.

Given the rapid industry adoption, Microsoft is coming up with industry standards for Cards, to define guidelines around seamless adoption to the host theme. For example, if you are using a black theme in your mail, then the card will automatically adopt this theme.

Here are some of the key benefits of Adaptive Cards

• Adaptive Cards are simple and declarative by design

• Cards are cross-platform compatible and easy to integrate

• Microsoft now offers Adaptive card templating, which allows creating, reusing, and sharing of adaptive cards. This helps in separating the data from the layout

• Conditional formatting enables the application of simple logic at the frontend

Automatic 'Image to Card' conversion: Why is this transformational?

Our journey with Adaptive Cards started when the BigThinkCode team designed and built a card renderer for the React Native community. At that time, Matt Hidinger, Principal Program Manager, Microsoft had quoted "We're pleased to seeBigThinkCode'scommitment to the Microsoft Adaptive Cards project and to recognize BigThinkCode as the official Maintainer & contributors ofReact Native Adaptive cardsand Pic2Card”

We have been collaborating with the Microsoft team since then and actively contributed to defining the Adaptive Cards roadmap. The deeper understanding of the architecture gained through the feature contributions has helped us to adopt this technology successfully for our customer projects. For a large retail major, we built a LUIS-based chatbot plugin designed to be integrated with any native application, with minimal setup. We achieved 'Multiple Canvas | Multiple Application' by using React Native Adaptive Cards for rendering the chatbot responses in a rich and dynamic conversational interface.

While templating gives the flexibility to reuse and share a card design, we wanted to make the process of creating cards even simpler by offering an option to directly convert a design image to a card. Using a combination of Computer Vision, Deep Learning, and Heuristic Models we have built an “Image to Card” conversion feature which can understand the design objects and their structure. As a result, the user can just upload an image and receive the card output in the JSON format effortlessly.

And the big news is here – the Microsoft team has integrated this exciting new feature with the source code, and they have displayed this at the Microsoft Build Conference 2020.