Remotely connected business & APIs
What is a “remotely connected business”? Although this is a definition of my choosing, there have been a lot of articles around the idea of the “connected company” or the “remote company”.
In my mind, the “remotely connected business” connects data and information from remote sources, and leverages this to generate new opportunities and improve existing products, systems and operations.
Companies need to expand their view on their environment by integrating external data. This is makes them more agile and competitive.
For most companies, data is gold and is typically kept as private as possible in order to be used as a competitive advantage. Yet, some companies have found ways to monetise this gold by making this data available to others. Previously isolated and private data sources are now accessible to all through tools like APIs. Businesses have been able to make data a differentiator or value add to their services and products.
At the time of writing, businesses across all industries are going through difficulties due to the unknowns and uncertainties that the COVID-19 outbreak is bringing to the economy.
In this situation, businesses need to be more “remotely connected” than ever, bringing as much of this external information as possible together in order to improve the view of their business and be ready for any changes in the environment. Businesses need BI leverage to move with agility and take action.
Building an Automated Market Intelligence Dashboard with APIs
In order to explore the idea of remotely connected businesses and the potential behind this concept, I was wanted to explore how a company can build an automated sales pipeline but also get a better overview of the market and how they compete using a mix of internal and external data. This exercise will show how external data can be used to stay competitive and generate new sales remotely.
For this, I mainly used the following 3 tools:
Domo is cloud platform to modernise any business process. It enables to connect, prepare, store and visualise your data in place place — and start to answer business questions immediately. Data is easily integrated through multiple external APIs, data uploads and scheduled email reports. Domo also has a trial version!
RapidAPI is a tool to get quick access to a large list of public and private APIs and generate authentification tokens. A lot of APIs can be access with their free or freemium options.
Data -miner is a tool to generate CSV exports of data by scrapping websites that don’t have easily accessible APIs. Their free plan is very generous with the ability to scrape 500 pages per month.
For this exercise, I have explored the idea of a UK Fintech SaaS payment company trying to ensure the continuity of its sales pipeline and market intelligence activities during the outbreak as sales and marketing teams may no longer be able to generate leads or gather information by going to events or face to face meetings.
Step 1 — Connect the data
In this exercise, I have used data from the following sources:
- Crunchbase data: Crunchbase is a platform for finding business information about private and public companies. In this case, the data provided by its API will be useful to identify the size of the market and competitors.
- Builtwith data: Builtwith tracks what technologies are being used by other websites, which can enable companies to enhance their prospect list or add metadata to current prospects in terms of technologies already in place.
- Angel.co data: Like Crunchabse, Angel.co keeps a record of the start-ups on several industries and the total amount of money raised by them. This is useful to enhance the information about the market and new comers not yet identified in Crunchbase.
- Yahoo Finance data: While Yahoo Finance has an API, for this use-case, as I was looking at a very small list of stocks, I decided to export the data in CSV and import it into Domo to get latest market valuation of the companies listed on the market.
- CRM Sales data: In this case, I generated a fake list of sales pipeline data based on the top websites in the UK as well as some random websites from the prospect list.
For the Crunchbase and Builtwith data, as they both offer public APIs, I was able to use the Domo JSON no code connector. This enabled me to make queries to the API and get a list of companies or websites in order to generate the prospect and competition datasets. As the Crunchbase API is limited when using the free option, I also imported some data into Domo using the email connector with data exported manually from Crunchbase.
I used the same approach for Builthwith using the API documentation provided on their website.
For Angel.co, I used public recipes on the Data-miner tool in order to scrape the list of companies on their websites. It is also possible to have these jobs scheduled and exported them automatically. Once in a CSV format, it is very easy to upload them into Domo the using drag and drop interface or Workbench application. The benefit of the Domo Workbench application is that then uploads can be automated when the file is updated on my computer.
Step 2 — Prepare the data
Once I had all the data in Domo, I could use the built in ETL tools to clean and join my different datasets. Domo has a MySQL tool that enables coders to script these manipulations, however, for ease of use, I used the Magic ETL option — which creates a workflow of the data preparation in a drag and drop style interface.
The dataflows in Domo enabled me to create 3 datasets:
- Competition List dataset: This includes all information from Crunchbase, Algel.co and Yahoo Finance on the companies in the Fintech, Payment and API industries.
- Prospect List dataset: Enriching my internal sales pipeline with the list of companies and websites from Builtwith using tools from the competition.
- Enhanced Prospect List dataset: Same as above but is joined with the prospect with competitor’s info.
The purpose of the above 3 datasets was to enable me to answer the following key questions that I was unable to answer with only internal data:
At each stage of the sale, who are we competing with?
How big is the market in terms of sales value? How much of this do we have identified in our sales pipeline?
Are there industries we should focus more based on estimated revenue?
Step 3 — Explore the data and create a dashboard
With the data from my CRM, I was quite limited in terms of pipeline — and dependant on the Sales team to add data to the CRM. Now, with the automated import from Builtwith, I can automatically add new identified prospects using competitor products.
As you can see above, adding the Sales pipeline has surfaced a lot more opportunities that can be reviewed and explored by the Sales team. The Builtwith data also comes with some links to the websites and emails.
This connection also enabled the ability to identify some new verticals or opportunities within already existing verticals where won deals could be used as success stories.
Finally, thanks to the competition data, I can have a better view of the composition of the market, total market valuation and an estimate of the revenue of the top competitors based on my prospect list.
It is also possible to using this method to analyse the data for global strategy (marketing slogan, HR…).
I was able to build the above dashboards in Domo in a single afternoon. I spent a day of work to get access to the data using the different APIs and to prepare the data in the ETL.
My work could be enhanced by using the paid APIs and thus getting access to the full range of data from the different sources. Yet, these dashboard can already be automated daily with the scheduling capabilities of Domo and new prospects and competitors would appear as they are added in Builtwith or Crunchbase.
Now that we don’t have the opportunity to go out and meet clients, prospects and analysts, I believe that businesses can leverage tools like Domo, RapidAPI and Data-miner to get access to data remotely and become “remotely connected”.
—
Story originally published on Medium.
