Tag: Data


Deprecated: Function create_function() is deprecated in /home2/thebrand/public_html/wp-content/plugins/wp-spamshield/wp-spamshield.php on line 1884
1-javascript-data-grid
15 Jul

You Can Do That With A JavaScript Data Grid?

by TBSCategories: News

Data grids, also known as data tables, are essential in presenting massive amounts of data to users. Users should be able to view the data in a way that’s easy to understand, analyze, and manipulate. However, building data grid views with performance, speed, and user experience in mind can be a particularly daunting task. This is especially true when building them from scratch or using libraries with limited functionality and sub-par performance.

There is no shortage of libraries that bundle data grids. However, most only offer a limited set of grid features, common among them being pagination, filtering, sorting, and theming. Other data grid libraries are built as wrappers that rely on several dependencies. These kinds of libraries unfavorably impact the performance of your grid compared to their native counterparts. They are not built anew for every framework or language. As such these non-native libraries can be slow, may fail to take advantage of superior features of a framework/language, lack crucial functionality, and require additional setup to get working.

Another thing these libraries are characterized by is poor user experience. They often fail to implement responsive design for different screen sizes and orientations, are unable to lock or make parts of a grid sticky, and make accessibility an afterthought. Besides that, they only provide editing in forms separate from the grid, which often involves multiple actions to complete. This can be tiring and repetitive notably when editing numerous data items. Others don’t even offer editing. To add to this, they tend to lack data export functionality and leave users relying on web page printing for exports.

Due to their limited functionality and features, you have to supplement them with separate libraries to build an adequate grid. For example, to chart data, you’d have to use a different chart library since the grid library won’t offer it. Moreover, you’re unable to embed these unrelated components in the grid since support for them is not in-built.

To address these problems, you’d have to use a library that’s not only built to be native but also incorporates a range of complementary components and focuses on great user experience and performance. To demonstrate the features of an ideal data grid, we’ll use Kendo UI Data Grids as an example. These data grids are one of 100+ components available in a library bundle called Progress® Kendo UI®. The bundle consists of four component libraries built natively for several frontend frameworks. These are Kendo UI for Angular, KendoReact, Kendo UI for Vue, and Kendo UI for jQuery. The examples given throughout this piece will feature grids from all four of these libraries.

Responsive Design

When it comes to data grids, your users must have a full view of the data they are working with. Data that is hidden or difficult to access is frustrating to read and turns users completely off your grid. A lot of grid libraries do not make their grids responsive and it’s up to you to implement it using styling and some logic. This can be especially complicated with data containing very many columns. If you are building multiple grids with different types of data with varying representation needs, this further compounds the complexity. You have to figure out scrolling, media queries, font sizes, scaling, whether to omit some parts of the data, and so on.

Modern data tables should be able to respond to changes in orientation and display all data well on all screen sizes. For example, the Kendo UI Data Grids adjust their size depending on the viewport size and the number of rows it accommodates. For example, in the Angular Grid, you can set its height and the grid will become scrollable if some of its contents do not fit. Setting the height only involves specifying a value for the grid’s height CSS property and ensuring that the parent element also has a height set. No other configuration is required. You can see how this is done in this sample stock table here.

Besides that, you can choose to toggle the visibility of the columns in the grid while still displaying all required data. You achieve this by creating different columns for different screen size ranges and using the media property on a column to decide where to show them. For instance, in this Angular data table, for larger screen sizes (media="(min-width: 450px)"), the columns are on full display and look like this.

However, you can choose to hide the price, in-stock, and discontinued columns on medium displays (media="(min-width: 680px)"). This should look like this:

On smaller displays ( media="(max-width: 450px)" ) , you can create a single custom column to show all the data similar to this:

Kendo UI Data Grids also support Bootstrap 4 device identifiers like xs, sm, md, lg, and xl. Although this is easier to use, it’s not as versatile since it limits the number of queries you can include to one. For instance, with your own breakpoints you could have something like media="(min-width: 500px) and (max-width: 1200px)". Combining multiple identifiers is not possible with Bootstrap 4 device identifiers.

Accessibility Compliance

Making sure that your grid meets modern accessibility standards should be a priority. Doing this ensures that people with disabilities can engage with your grid and guarantees that there is equity among your users. Still, some libraries do nothing to make sure their grids are accessible. Others do only the bare minimum resulting in sub-standard grids when evaluated for accessibility. Augmenting these grids to be accessible involves a fair amount of work. This is further complicated by more intricate grid designs. Although this work will pay off later for you and your users, these libraries should have made accessibility a core part of their products.

Kendo UI Data Grids prioritize it by supporting the main accessibility standards like WAI-ARIA, Section 508, and WCAG 2.1. For example, KendoReact follows the Section 508 standard by ensuring that most of its components are completely accessible and support keyboard navigation. It follows WCAG’s Keyboard Accessible guideline by making the grid and all its embedded components keyboard operable. As a result, the React Grid achieves the highest WCAG conformance level of AAA. Being a web component, KendoReact Data Grid fulfills the WAI-RAI specification to ensure that users with disabilities can adequately interact with it on web pages. In this React data grid, for example, you can navigate to the different components and rows using a keyboard.

Virtual Scrolling

With virtual scrolling, only a segment of data is rendered within the grid. This is usually set as a number of records to fetch. When a user scrolls past this segment, another one of the same size is rendered. This helps with performance as rendering a large data set takes up a lot of memory and hobbles the performance and speed of your grid. Virtual scrolling gives the illusion of rendering all the data without any of the performance consequences.

Virtual scrolling is not often supported by grid libraries. Instead, they encourage pagination which may not be the best experience for users when viewing massive amounts of data. When attempting to render enormous data quantities, the grid’s performance suffers further contributing to a terrible user experience. For libraries that support virtual scrolling, it only applies to records in the data and not specific parts of the records. This is particularly limiting when the data has several columns.

Kendo UI supports virtual scrolling for both local and remote data. For example, in the Kendo UI for jQuery Grid, you enable it by setting the scrollable.virtual property of a grid to true. By setting this, the grid only loads the number of items specified by the pageSize property of the grid data source. You can see how this works in this jQuery data grid which uses local data. 

<!DOCTYPE html> <html>   <head>...</head>   <body>     ...     <div id="grid"></div>     <script>       var dataSource = new kendo.data.DataSource({                             pageSize: 20,               ...             });       $ ("#grid").kendoGrid({                         dataSource: dataSource,                         scrollable: {                             virtual: true                         },         ...       });     </script>   </body> </html>

This same setting will work for remote data as seen in this jQuery data table. Additionally, you can use a similar setting to virtualize the columns of a grid if records contain several properties that may be costly to render all at once. The scrollable.virtual property needs to be set to true. However, virtualizing columns does not depend on the pageSize property. This example demonstrates this feature.

PDF And Excel Exports

Having the ability to export data from the grid is pivotal. Users may need to distribute or further manipulate it using applications like spreadsheets. Your users should have the option to painlessly share data without being confined to the grid. Grid data may also need extra processing not offered by your grid, like in spreadsheets and presentation software.

Although this is an essential use case, it is not catered for in many libraries. Users have to resort to printing whole web pages to get access to the data in PDF formats. When transferring data to external applications, they have to copy and paste it numerous times. This is understandably pretty infuriating.

Kendo UI Data Grids provide data exports from the grid in two formats: PDF and Excel. For instance, in the Kendo UI for Vue Data Grid, to process PDF exports, you would use the GridPDFExport component. With its save method, you would pass the data you’d like to include in the PDF export. The data could be paginated or the complete set. 

<template>     <button @click="exportPDF">Export PDF</button>     <pdfexport ref="gridPdfExport">         <Grid :data-items="items"></Grid>     </pdfexport> </template> <script> import { GridPdfExport } from '@progress/kendo-vue-pdf'; import { Grid } from '@progress/kendo-vue-grid';  export default {     components: {         'Grid': Grid,         'pdfexport': GridPdfExport     },     data: function () {         return {             products: [],             ...         };     },     methods: {         exportPDF: function() {             (this.$ refs.gridPdfExport).save(this.products);         },        ...     },     ... }; </script>

The GridPDFExport component allows you to specify page sizes for the export, page margins, how to scale the grid on the page, etc. This is useful for customizing larger grids to fit the PDF pages. You would pass these as properties to the component. Here’s an example:

<pdfexport ref="exportPDF" :margin="'2cm'" :paper-size="'a4'" :scale="0.5">         <Grid :data-items="products"></Grid> </pdfexport>

You may choose to further customize the export using a template. Within the template, you can add styling, specify headers and footers, change the layout of the page, and add new elements to it. You would use CSS for styling. Once you’re done configuring the template, you would specify it using the page-template property of the GridPDFExport component.

To export Excel files from a Kendo UI Vue Grid, you would use the ExcelExportcomponent. With its saveExcel method, you pass the file name, the grid data, columns to display, etc. to it and call the method to generate the file. This Vue data grid is a great example of how you can achieve Excel exports with Kendo UI Vue Grid.

Sticky Columns

When a user scrolls through a grid horizontally, they may need to have some columns frozen or constantly within view. These columns usually contain crucial information like IDs, names, etc. Frozen/sticky columns always remain visible but may move either to the left or right edges of the grid depending on your scroll direction, or not move at all. For example, in this Vue data grid demo, the ID is frozen and the Discontinued column is sticky.

Sticky columns in grid libraries can be a rare occurrence. If not present, implementing it from scratch can be a difficult endeavor. It will require significant styling to accomplish and it may not scale well if you need numerous grids.

Setting up sticky columns in Kendo UI requires minimal setup. For instance, in a Kendo UI Vue Grid, you’ll need to set the locked property of a column to true to make it sticky. In this Vue data table, the ID and Discontinued columns are made sticky by setting the locked property. In the example below, the ID and Age are locked. 

<template>     <grid :data-items="people" :columns = "columns">     </grid> </template> <script> import { Grid } from '@progress/kendo-vue-grid'; import { people } from './people'  export default {     components: {         'grid': Grid     },      data: function () {         return {             people: this.getPeople(),             columns: [                 { field: 'ID', title: 'ID', locked: true},                 { field: 'FirstName', title: 'FirstName' },                 { field: 'LastName', title: 'LastName' },                 { field: 'Age', title: 'Age', locked: true},             ]         };     },     methods: {         getPeople() {            return people;         }     } }; </script>

Editing

A grid’s main use case is to view large amounts of data. Some libraries just stick to this and don’t consider the possibility that editing is needed. This disadvantages users as editing is a pretty useful feature. When users request it, developers are then forced to create a separate page for editing individual entries. To add to this, users are only able to edit one entry after another on one form. This is wearisome and makes for a bad user experience especially when handling large amounts of data.

One important use case for grid editing is facilitating batch editing. It’s useful for modifying large amounts of data all at once. This could involve deleting, creating, and updating the data.

Kendo UI Data Grids enables editing in two forms: inline and using pop-ups. With inline editing, all the data is edited within the grid. When a cell is clicked, it becomes editable. In a pop-up, a pop-up form is used to edit each entry individually. In this Kendo UI for jQuery table example, making a grid editable involves three steps: setting the grid’s editable configuration, establishing a data source, and configuring CRUD operations on the data source. These few steps reduce the complexity involved in setting up batch editing. Configuring pop-up editing follows the same steps but with different options at the start.

In addition to supporting edits, the Kendo UI for jQuery Grid enables input validation. For example, you can make inputs required or enforce a minimum length. Besides that, you can create custom input controls. Input controls are not just limited to text fields. You can use drop-downs, checkboxes, date pickers, range sliders, etc. These can be both inline and in pop-ups. In this jQuery data table, the Category field is a drop-down. Validation is also demonstrated in the same example. The unit price field has validation imposed, ensuring its least value is 1.

Supplementary Components

Most grid libraries have a singular purpose: to provide a grid. They do not ship with anything else, only the grid. You are limited to the features it provides. If you need to supplement the grid, it can be tricky because other libraries may not be compatible with it. So you just have to stick within the boundaries of the library when building a grid.

Kendo UI solves this because the approach of its creator is to offer a comprehensive library of components that easily integrate with each other instead of single components. The grid is part of a library of numerous components that allow you to do everything from data management, navigation, charting, editing, media presentation, chat facilitation,, and so on. You can choose to embed these components within the grid without having to perform some elaborate setup and possibly breaking it. Integrating them is seamless and requires minimal configuration. Take for example this Angular data table, its 1 Day column embeds a fully interactive chart for each row seamlessly. You can embed any number of components within a grid trusting that it will work and have all its features perform as expected.

Conclusion

Data grids need to be easy to understand, engaging, responsive, and accessible. They need to perform well and load data fast. However, building a data grid that meets these standards from scratch can take a long time and be a huge undertaking. You may opt to use data grid libraries but often these are not optimized for performance, are not accessible, and only ship with a single grid component.

Creating an appealing data grid that’s delightful to use requires a library that focuses on performance. It can do this by building natively and supporting virtual scrolling. The data grid it provides needs to be responsive and use sticky columns. This is so users can easily view the data no matter the screen size or orientation. Accessibility should be a core concern of grids. This guarantees that all users can have an equal experience using them.

Data tables should expand what a user can do with the data. This can be achieved through editing and facilitating exports in multiple formats. Besides that, these libraries should ship with other components to supplement the grid. Having compatible components in one library removes the need to use several different conflicting libraries in one application. A data grid library that provides these features will help you craft a great product for your users without much complication.

Articles on Smashing Magazine — For Web Designers And Developers

10 Dec

How To Overcome Data Onboarding Challenges For Software Products

by TBSCategories: News

Companies willing to pay good money for a new piece of software are most likely not starting from scratch. They’re running an established business, with well-built and documented processes. So, they have tons of data to carry over.

As a result, the decision to bring a new app into the fold is not one they take lightly. Internal processes need to change. Getting the team to adopt the new solution can take time. Integrating it with existing systems and external tools can be a problem. Oh yeah, and there’s the matter of compliance to worry about, too.

This means there’s a lot of pressure on new software to provide a top-notch experience from the get-go. Fail to provide companies with a simple and intuitive way to onboard their data and you can expect high rates of customer churn as a result.

If you’re designing a product that needs data from customers in order to be of any value, here’s what you need to know about building out your data onboarding process.

How Data Onboarding Correlates With User Satisfaction

Business software is essentially just an empty box waiting to be filled with its users’ data. Without the ability to flawlessly onboard users’ data, the software essentially becomes useless.

Let’s look at what happens when you get the data onboarding process right.

End User Benefits

If you can nail the data onboarding piece, expect your end users to reap the following benefits:

  • They’ll be more confident in their decision.
    With complete and accurate data transferred into your software, users actually see how valuable it is soon after signing up. This leaves little room for second-guessing their decision, which leads to greater satisfaction overall with the product, and ultimately more money for your business.
  • You’ll get greater team buy-in.
    A positive data onboarding experience lets customers use your product faster, reducing the time needed for them to get value. So, really, data onboarding sets the stage for how your customers and their team will view the rest of your app.
  • They’ll experience more success with the software.
    Since users won’t have to stress about data formatting and cleanup or troubleshooting error-ridden import processes, they can get more out of the product and its features.

Software Developer Benefits

The software provider (you and/or your client) benefits, too:

  • Improve user satisfaction.
    Your end users don’t need to be technical wizards to figure out how to onboard data into your product. When you make light work of this, you reduce churn, attract more users and retain more loyal users over the long-term.
  • Spend less time on customer service.
    You can stop worrying about having to support a faulty data onboarding process as well as taking over tasks like data formatting and validation for your users. Instead, put your time and energy towards building better relationships with customers instead of putting out fires all the time.
    Kelly Abbott, Co-Founder and CTO of Tablecloth, can attest to this:

    “We have cut the amount of time we spend wrangling with files by 95%. We basically had all hands working to solve those problems at times.”

  • Have greater confidence in your product.
    When you have a data onboarding solution that’s flexible and powerful, you don’t have to restrict what data your users can or can’t import. It’s no longer a limitation.
    As Abbott explains:

    “It has made us more contemplative about the data we are asking clients for. We no longer have to avoid asking for data that may require too much time to fix. Flatfile eliminates that problem and has improved our willingness to experiment with different types of data we can incorporate into our analyses. The more time we spend tinkering with different data types, the more likely we are going to uncover the insight that produces additional value in the marketplace. That is indispensable for a startup like us.”

  • Save money.
    Although you’ll have to spend money on a third-party data onboarding solution, you’ll save your company the time and money otherwise spent trying to manage a custom-built data importer, onboarding process and client relationships. (Tablecloth, for instance, saved tens of thousands of dollars when they adopted Flatfile.)

The Challenges Of Data Onboarding For Software Products

Let’s have a look at the common challenges in data onboarding and how Flatfile Concierge removes them:

Challenge #1: There’s A Lot Of Data To Aggregate

When signing up for new business software, users probably expect to do a little work upfront, like filling in basic account information, configuring settings and adding users. The last thing you want to do is surprise them with a data importer that’s going to cause more work for them.

Let’s say, for example, you’ve built a CRM.

Unless the software targets startups and other new businesses, users are going to have a ton of external data to bring along. For instance:

  • Contact info for clients, prospects, vendors, partners and team members;
  • Existing customer data like account and sales history;
  • Prospect data like communication history;
  • Sales pipeline details;
  • Team and individual goals and metrics.

Unless your CRM directly integrates with every one of your users’ previous CRMs, how are they going to move this data over? Copy and paste? CSV templates?

Concierge allows companies to import ‘data model’ templates based on spreadsheet columns for easy setup. Customers would then import data that automatically matches this model with no code needed. (Image source: Flatfile)

Plus, you have to think about all of the other sources a CRM pulls info in from. Payment gateways. Spreadsheets that live on a sales team’s drive. Signed contracts that have been emailed or faxed to your company. There’s a lot of data coming from different places and people.

The Fix

There are a number of things Flatfile Concierge does to fix this problem.

For starters, it allows data to be imported from a variety of file types:

  • CSV,
  • TSV,
  • XLS,
  • XML,
  • And more.

With this kind of flexibility, your users won’t have to worry about transferring data to one specific file type and then cleaning up errors that occur during the transfer. Flatfile Concierge can handle various file types, of varying data types, and easily validate it all.

Another thing to think about is how your software is going to track and organize each imported file and its corresponding data.

What Flatfile allows your users to do is create collaborative workspaces to place data in. When a team member adds new data to the workspace, a record is captured containing the:

  • Date of upload,
  • File name,
  • User who submitted the data,
  • Number of rows added,
  • Version history,
  • Upload errors.

Flatfile Concierge animation demonstrating notifications for when spreadsheets are imported. (Image source: Flatfile)

This will keep things organized while also keeping everyone accountable to the data they contribute. And with this information readily available from a centralized dashboard, there’ll be no secret as to what’s been uploaded, by whom and when. Import errors can also be fixed collaboratively, without the need to re-upload spreadsheet data.

Challenge #2: Data Is Imported In A Variety Of States

When you give your software users the ability to transfer their data into your product, there’s not a lot you or the software team can do in terms of formatting or cleaning up end users’ data beforehand. Nor should you have to. Your job is to ensure customers see the value in the software; not to struggle with importing data.

You could give them a spreadsheet template, but that would require them to spend time reformatting all their data. You could point them to the knowledgebase, but, again, that assumes that your end users will be willing to do that extra work.

In reality, your users are going to be in a hurry to get inside the new software and get to work. They’re not going to stop to deal with this. That’s the software’s job.

However, many data onboarding solutions don’t handle messy spreadsheets very well. Not only do they have a hard time recognizing what some of the data is (often because the data model doesn’t match their own), but then the application refuses to accept certain spreadsheet columns.

Even if it’s the end user’s fault for not properly organizing or labeling their data or teaching their team how to do so (or just not knowing what to do in the first place), who do you think they’re going to blame in the end when their data won’t import?

The Fix

Flatfile Concierge’s importer is AI-powered, which means that your software (and data importer) really can do the work for your end users.

Using advanced validation logic, the data importer can figure out what the data is and where it goes.

While Flatfile will automatically match columns and corresponding data to your software’s actual data fields, users get a chance to confirm that’s the case before allowing it into the system:

Before this happens, you can do a little work on the backend to ensure that Flatfile knows what to do with your users’ data:

  • Create target data models so Flatfile can navigate complex spreadsheet formats and datatypes your users will likely try to import.
  • Create a template with validation rules so Flatfile’s AI knows exactly how to map everything out.
  • Validate imported data against other databases to help the importer contextualize, validate and clean up the data over time.

Once you’ve done that upfront work, the rest is easy.

The bulk of the work will be done by Flatfile Concierge when it transforms imported data into something clean and useful. In fact, about 95% of imported columns will automatically map to your software thanks to Flatfile’s machine learning and fuzzy matching system.

The end user will have the opportunity to review the parts of their data that contain errors. If they find any, they can repair the errors inside Flatfile, rather than have to fix it in a spreadsheet and re-import.

Challenge #3: Getting And Tracking Data From Multiple Users

When there are a lot of cooks in the kitchen, there are a number of things that can go wrong.

Data can sometimes live on team members’ computers, or worse, sent over email, which can be a huge security concern for sensitive data. This can happen if users aren’t given access to the software platform or find the data importer too intimidating to use.

On the flip side of that, with the wrong data onboarding process, it could become like a free-for-all where people add whatever the heck they like to the company’s data. While the data does get imported, there’s no review framework so the company’s database is filled with errors and duplicate entries.

Your end users need to be able to maintain order, control and security when dealing with something as serious as company data — especially if you want your software to be usable.

The Fix

Flatfile Concierge has designed the data onboarding process to be a collaborative one.

As you can see, company admins can invite specific collaborators (i.e. customers) to add data to their workspaces. But this isn’t a blanket invitation to import data.

Admins have the ability to create an approval process. They get to:

  • Ask for specific data sets from team members.
  • Control which workspaces they’re allowed to import data to.
  • Review all data submissions before flowing the approved data into the platform.

Admins can also import data on the customer’s behalf. Flatfile Concierge ensures that data onboarding is never a dead-end for customers.

Not only does this ensure that the right data ends up in the software, but the controlled flow means the data will end up being cleaner and more accurate, too. All of this, while providing a seamless data onboarding experience for users.

Challenge #4: Data Security Is Always A Concern

When it comes to web and app development, user privacy and security are top priority. If our customers and visitors don’t trust that their information is safe from prying eyes (and isn’t being sold off to advertisers), they’re going to stop using our solutions in the first place.

The same thing happens with software — though it’s not just the company’s personal data they have to worry about securing.

Often, when companies import data into software (like the CRM example), they’re importing their customers’ private and sensitive data. Allow that to be compromised and you can kiss your software goodbye.

So, yes, the software itself needs to be secured. That’s a given. But so, too, does your data onboarding process. It’s a huge point of vulnerability if left unchecked.

The Fix

The first thing Flatfile Concierge does is to encourage users to move away from sharing sensitive data over email, FTP, and other unsecured platforms by providing a user-friendly data onboarding solution.

The second thing it does is provide an authenticated and compliant workspace for users to import, validate, and post their data to your software.

Here’s how Flatfile Concierge secures its workspaces:

  • Each collaborator enters the data importer through an authenticated invitation.
  • Data is encrypted in transit and stored in an encrypted Amazon S3 bucket.
  • The data onboarding platform is 100% GDPR compliant.
  • Flatfile is HIPAA and SOC2 compliant and can adjust for other compliance requirements as needed.

In addition, once data is successfully migrated into your application, it’s deleted from Flatfile. This way, you only have to worry about securing your data within your software and not on previous platforms it’s touched.

Wrapping Up

With an insufficient or error-prone data onboarding process, you, the software provider and its end users are going to spend too much time manually cleaning and validating spreadsheets. This won’t just happen during the initial user signup either. If the data importer isn’t up to the task, you’re all going to be throwing away a ton of time and resources every time data needs to be uploaded or transferred into the platform from existing customers.

Of course, this all assumes that your importer can even get user data into the software. (Sadly, this happens with too many custom-built solutions.)

Needless to say: Your data onboarding process must be flawless for your team and customers. It’s the only way to keep user churn rates low and user satisfaction high.

Data onboarding is a really complex process to handle. Save yourself the trouble in trying to develop your own data onboarding solution and the time trying to troubleshoot the problems with it. With an AI-powered data importer like Flatfile Concierge, everything’s taken care of for you.

Articles on Smashing Magazine — For Web Designers And Developers

09 Dec

How To Overcome Data Onboarding Challenges For Software Products

by TBSCategories: News

Companies willing to pay good money for a new piece of software are most likely not starting from scratch. They’re running an established business, with well-built and documented processes. So, they have tons of data to carry over.

As a result, the decision to bring a new app into the fold is not one they take lightly. Internal processes need to change. Getting the team to adopt the new solution can take time. Integrating it with existing systems and external tools can be a problem. Oh yeah, and there’s the matter of compliance to worry about, too.

This means there’s a lot of pressure on new software to provide a top-notch experience from the get-go. Fail to provide companies with a simple and intuitive way to onboard their data and you can expect high rates of customer churn as a result.

If you’re designing a product that needs data from customers in order to be of any value, here’s what you need to know about building out your data onboarding process.

How Data Onboarding Correlates With User Satisfaction

Business software is essentially just an empty box waiting to be filled with its users’ data. Without the ability to flawlessly onboard users’ data, the software essentially becomes useless.

Let’s look at what happens when you get the data onboarding process right.

End User Benefits

If you can nail the data onboarding piece, expect your end users to reap the following benefits:

  • They’ll be more confident in their decision.
    With complete and accurate data transferred into your software, users actually see how valuable it is soon after signing up. This leaves little room for second-guessing their decision, which leads to greater satisfaction overall with the product, and ultimately more money for your business.
  • You’ll get greater team buy-in.
    A positive data onboarding experience lets customers use your product faster, reducing the time needed for them to get value. So, really, data onboarding sets the stage for how your customers and their team will view the rest of your app.
  • They’ll experience more success with the software.
    Since users won’t have to stress about data formatting and cleanup or troubleshooting error-ridden import processes, they can get more out of the product and its features.

Software Developer Benefits

The software provider (you and/or your client) benefits, too:

  • Improve user satisfaction.
    Your end users don’t need to be technical wizards to figure out how to onboard data into your product. When you make light work of this, you reduce churn, attract more users and retain more loyal users over the long-term.
  • Spend less time on customer service.
    You can stop worrying about having to support a faulty data onboarding process as well as taking over tasks like data formatting and validation for your users. Instead, put your time and energy towards building better relationships with customers instead of putting out fires all the time.
    Kelly Abbott, Co-Founder and CTO of Tablecloth, can attest to this:

    “We have cut the amount of time we spend wrangling with files by 95%. We basically had all hands working to solve those problems at times.”

  • Have greater confidence in your product.
    When you have a data onboarding solution that’s flexible and powerful, you don’t have to restrict what data your users can or can’t import. It’s no longer a limitation.
    As Abbott explains:

    “It has made us more contemplative about the data we are asking clients for. We no longer have to avoid asking for data that may require too much time to fix. Flatfile eliminates that problem and has improved our willingness to experiment with different types of data we can incorporate into our analyses. The more time we spend tinkering with different data types, the more likely we are going to uncover the insight that produces additional value in the marketplace. That is indispensable for a startup like us.”

  • Save money.
    Although you’ll have to spend money on a third-party data onboarding solution, you’ll save your company the time and money otherwise spent trying to manage a custom-built data importer, onboarding process and client relationships. (Tablecloth, for instance, saved tens of thousands of dollars when they adopted Flatfile.)

The Challenges Of Data Onboarding For Software Products

Let’s have a look at the common challenges in data onboarding and how Flatfile Concierge removes them:

Challenge #1: There’s A Lot Of Data To Aggregate

When signing up for new business software, users probably expect to do a little work upfront, like filling in basic account information, configuring settings and adding users. The last thing you want to do is surprise them with a data importer that’s going to cause more work for them.

Let’s say, for example, you’ve built a CRM.

Unless the software targets startups and other new businesses, users are going to have a ton of external data to bring along. For instance:

  • Contact info for clients, prospects, vendors, partners and team members;
  • Existing customer data like account and sales history;
  • Prospect data like communication history;
  • Sales pipeline details;
  • Team and individual goals and metrics.

Unless your CRM directly integrates with every one of your users’ previous CRMs, how are they going to move this data over? Copy and paste? CSV templates?

Concierge allows companies to import ‘data model’ templates based on spreadsheet columns for easy setup. Customers would then import data that automatically matches this model with no code needed. (Image source: Flatfile)

Plus, you have to think about all of the other sources a CRM pulls info in from. Payment gateways. Spreadsheets that live on a sales team’s drive. Signed contracts that have been emailed or faxed to your company. There’s a lot of data coming from different places and people.

The Fix

There are a number of things Flatfile Concierge does to fix this problem.

For starters, it allows data to be imported from a variety of file types:

  • CSV,
  • TSV,
  • XLS,
  • XML,
  • And more.

With this kind of flexibility, your users won’t have to worry about transferring data to one specific file type and then cleaning up errors that occur during the transfer. Flatfile Concierge can handle various file types, of varying data types, and easily validate it all.

Another thing to think about is how your software is going to track and organize each imported file and its corresponding data.

What Flatfile allows your users to do is create collaborative workspaces to place data in. When a team member adds new data to the workspace, a record is captured containing the:

  • Date of upload,
  • File name,
  • User who submitted the data,
  • Number of rows added,
  • Version history,
  • Upload errors.

Flatfile Concierge animation demonstrating notifications for when spreadsheets are imported. (Image source: Flatfile)

This will keep things organized while also keeping everyone accountable to the data they contribute. And with this information readily available from a centralized dashboard, there’ll be no secret as to what’s been uploaded, by whom and when. Import errors can also be fixed collaboratively, without the need to re-upload spreadsheet data.

Challenge #2: Data Is Imported In A Variety Of States

When you give your software users the ability to transfer their data into your product, there’s not a lot you or the software team can do in terms of formatting or cleaning up end users’ data beforehand. Nor should you have to. Your job is to ensure customers see the value in the software; not to struggle with importing data.

You could give them a spreadsheet template, but that would require them to spend time reformatting all their data. You could point them to the knowledgebase, but, again, that assumes that your end users will be willing to do that extra work.

In reality, your users are going to be in a hurry to get inside the new software and get to work. They’re not going to stop to deal with this. That’s the software’s job.

However, many data onboarding solutions don’t handle messy spreadsheets very well. Not only do they have a hard time recognizing what some of the data is (often because the data model doesn’t match their own), but then the application refuses to accept certain spreadsheet columns.

Even if it’s the end user’s fault for not properly organizing or labeling their data or teaching their team how to do so (or just not knowing what to do in the first place), who do you think they’re going to blame in the end when their data won’t import?

The Fix

Flatfile Concierge’s importer is AI-powered, which means that your software (and data importer) really can do the work for your end users.

Using advanced validation logic, the data importer can figure out what the data is and where it goes.

While Flatfile will automatically match columns and corresponding data to your software’s actual data fields, users get a chance to confirm that’s the case before allowing it into the system:

Before this happens, you can do a little work on the backend to ensure that Flatfile knows what to do with your users’ data:

  • Create target data models so Flatfile can navigate complex spreadsheet formats and datatypes your users will likely try to import.
  • Create a template with validation rules so Flatfile’s AI knows exactly how to map everything out.
  • Validate imported data against other databases to help the importer contextualize, validate and clean up the data over time.

Once you’ve done that upfront work, the rest is easy.

The bulk of the work will be done by Flatfile Concierge when it transforms imported data into something clean and useful. In fact, about 95% of imported columns will automatically map to your software thanks to Flatfile’s machine learning and fuzzy matching system.

The end user will have the opportunity to review the parts of their data that contain errors. If they find any, they can repair the errors inside Flatfile, rather than have to fix it in a spreadsheet and re-import.

Challenge #3: Getting And Tracking Data From Multiple Users

When there are a lot of cooks in the kitchen, there are a number of things that can go wrong.

Data can sometimes live on team members’ computers, or worse, sent over email, which can be a huge security concern for sensitive data. This can happen if users aren’t given access to the software platform or find the data importer too intimidating to use.

On the flip side of that, with the wrong data onboarding process, it could become like a free-for-all where people add whatever the heck they like to the company’s data. While the data does get imported, there’s no review framework so the company’s database is filled with errors and duplicate entries.

Your end users need to be able to maintain order, control and security when dealing with something as serious as company data — especially if you want your software to be usable.

The Fix

Flatfile Concierge has designed the data onboarding process to be a collaborative one.

As you can see, company admins can invite specific collaborators (i.e. customers) to add data to their workspaces. But this isn’t a blanket invitation to import data.

Admins have the ability to create an approval process. They get to:

  • Ask for specific data sets from team members.
  • Control which workspaces they’re allowed to import data to.
  • Review all data submissions before flowing the approved data into the platform.

Admins can also import data on the customer’s behalf. Flatfile Concierge ensures that data onboarding is never a dead-end for customers.

Not only does this ensure that the right data ends up in the software, but the controlled flow means the data will end up being cleaner and more accurate, too. All of this, while providing a seamless data onboarding experience for users.

Challenge #4: Data Security Is Always A Concern

When it comes to web and app development, user privacy and security are top priority. If our customers and visitors don’t trust that their information is safe from prying eyes (and isn’t being sold off to advertisers), they’re going to stop using our solutions in the first place.

The same thing happens with software — though it’s not just the company’s personal data they have to worry about securing.

Often, when companies import data into software (like the CRM example), they’re importing their customers’ private and sensitive data. Allow that to be compromised and you can kiss your software goodbye.

So, yes, the software itself needs to be secured. That’s a given. But so, too, does your data onboarding process. It’s a huge point of vulnerability if left unchecked.

The Fix

The first thing Flatfile Concierge does is to encourage users to move away from sharing sensitive data over email, FTP, and other unsecured platforms by providing a user-friendly data onboarding solution.

The second thing it does is provide an authenticated and compliant workspace for users to import, validate, and post their data to your software.

Here’s how Flatfile Concierge secures its workspaces:

  • Each collaborator enters the data importer through an authenticated invitation.
  • Data is encrypted in transit and stored in an encrypted Amazon S3 bucket.
  • The data onboarding platform is 100% GDPR compliant.
  • Flatfile is HIPAA and SOC2 compliant and can adjust for other compliance requirements as needed.

In addition, once data is successfully migrated into your application, it’s deleted from Flatfile. This way, you only have to worry about securing your data within your software and not on previous platforms it’s touched.

Wrapping Up

With an insufficient or error-prone data onboarding process, you, the software provider and its end users are going to spend too much time manually cleaning and validating spreadsheets. This won’t just happen during the initial user signup either. If the data importer isn’t up to the task, you’re all going to be throwing away a ton of time and resources every time data needs to be uploaded or transferred into the platform from existing customers.

Of course, this all assumes that your importer can even get user data into the software. (Sadly, this happens with too many custom-built solutions.)

Needless to say: Your data onboarding process must be flawless for your team and customers. It’s the only way to keep user churn rates low and user satisfaction high.

Data onboarding is a really complex process to handle. Save yourself the trouble in trying to develop your own data onboarding solution and the time trying to troubleshoot the problems with it. With an AI-powered data importer like Flatfile Concierge, everything’s taken care of for you.

Articles on Smashing Magazine — For Web Designers And Developers

09 Dec

How To Overcome Data Onboarding Challenges For Software Products

by TBSCategories: News

Companies willing to pay good money for a new piece of software are most likely not starting from scratch. They’re running an established business, with well-built and documented processes. So, they have tons of data to carry over.

As a result, the decision to bring a new app into the fold is not one they take lightly. Internal processes need to change. Getting the team to adopt the new solution can take time. Integrating it with existing systems and external tools can be a problem. Oh yeah, and there’s the matter of compliance to worry about, too.

This means there’s a lot of pressure on new software to provide a top-notch experience from the get-go. Fail to provide companies with a simple and intuitive way to onboard their data and you can expect high rates of customer churn as a result.

If you’re designing a product that needs data from customers in order to be of any value, here’s what you need to know about building out your data onboarding process.

How Data Onboarding Correlates With User Satisfaction

Business software is essentially just an empty box waiting to be filled with its users’ data. Without the ability to flawlessly onboard users’ data, the software essentially becomes useless.

Let’s look at what happens when you get the data onboarding process right.

End User Benefits

If you can nail the data onboarding piece, expect your end users to reap the following benefits:

  • They’ll be more confident in their decision.
    With complete and accurate data transferred into your software, users actually see how valuable it is soon after signing up. This leaves little room for second-guessing their decision, which leads to greater satisfaction overall with the product, and ultimately more money for your business.
  • You’ll get greater team buy-in.
    A positive data onboarding experience lets customers use your product faster, reducing the time needed for them to get value. So, really, data onboarding sets the stage for how your customers and their team will view the rest of your app.
  • They’ll experience more success with the software.
    Since users won’t have to stress about data formatting and cleanup or troubleshooting error-ridden import processes, they can get more out of the product and its features.

Software Developer Benefits

The software provider (you and/or your client) benefits, too:

  • Improve user satisfaction.
    Your end users don’t need to be technical wizards to figure out how to onboard data into your product. When you make light work of this, you reduce churn, attract more users and retain more loyal users over the long-term.
  • Spend less time on customer service.
    You can stop worrying about having to support a faulty data onboarding process as well as taking over tasks like data formatting and validation for your users. Instead, put your time and energy towards building better relationships with customers instead of putting out fires all the time.
    Kelly Abbott, Co-Founder and CTO of Tablecloth, can attest to this:

    “We have cut the amount of time we spend wrangling with files by 95%. We basically had all hands working to solve those problems at times.”

  • Have greater confidence in your product.
    When you have a data onboarding solution that’s flexible and powerful, you don’t have to restrict what data your users can or can’t import. It’s no longer a limitation.
    As Abbott explains:

    “It has made us more contemplative about the data we are asking clients for. We no longer have to avoid asking for data that may require too much time to fix. Flatfile eliminates that problem and has improved our willingness to experiment with different types of data we can incorporate into our analyses. The more time we spend tinkering with different data types, the more likely we are going to uncover the insight that produces additional value in the marketplace. That is indispensable for a startup like us.”

  • Save money.
    Although you’ll have to spend money on a third-party data onboarding solution, you’ll save your company the time and money otherwise spent trying to manage a custom-built data importer, onboarding process and client relationships. (Tablecloth, for instance, saved tens of thousands of dollars when they adopted Flatfile.)

The Challenges Of Data Onboarding For Software Products

Let’s have a look at the common challenges in data onboarding and how Flatfile Concierge removes them:

Challenge #1: There’s A Lot Of Data To Aggregate

When signing up for new business software, users probably expect to do a little work upfront, like filling in basic account information, configuring settings and adding users. The last thing you want to do is surprise them with a data importer that’s going to cause more work for them.

Let’s say, for example, you’ve built a CRM.

Unless the software targets startups and other new businesses, users are going to have a ton of external data to bring along. For instance:

  • Contact info for clients, prospects, vendors, partners and team members;
  • Existing customer data like account and sales history;
  • Prospect data like communication history;
  • Sales pipeline details;
  • Team and individual goals and metrics.

Unless your CRM directly integrates with every one of your users’ previous CRMs, how are they going to move this data over? Copy and paste? CSV templates?

Concierge allows companies to import ‘data model’ templates based on spreadsheet columns for easy setup. Customers would then import data that automatically matches this model with no code needed. (Image source: Flatfile)

Plus, you have to think about all of the other sources a CRM pulls info in from. Payment gateways. Spreadsheets that live on a sales team’s drive. Signed contracts that have been emailed or faxed to your company. There’s a lot of data coming from different places and people.

The Fix

There are a number of things Flatfile Concierge does to fix this problem.

For starters, it allows data to be imported from a variety of file types:

  • CSV,
  • TSV,
  • XLS,
  • XML,
  • And more.

With this kind of flexibility, your users won’t have to worry about transferring data to one specific file type and then cleaning up errors that occur during the transfer. Flatfile Concierge can handle various file types, of varying data types, and easily validate it all.

Another thing to think about is how your software is going to track and organize each imported file and its corresponding data.

What Flatfile allows your users to do is create collaborative workspaces to place data in. When a team member adds new data to the workspace, a record is captured containing the:

  • Date of upload,
  • File name,
  • User who submitted the data,
  • Number of rows added,
  • Version history,
  • Upload errors.

Flatfile Concierge animation demonstrating notifications for when spreadsheets are imported. (Image source: Flatfile)

This will keep things organized while also keeping everyone accountable to the data they contribute. And with this information readily available from a centralized dashboard, there’ll be no secret as to what’s been uploaded, by whom and when. Import errors can also be fixed collaboratively, without the need to re-upload spreadsheet data.

Challenge #2: Data Is Imported In A Variety Of States

When you give your software users the ability to transfer their data into your product, there’s not a lot you or the software team can do in terms of formatting or cleaning up end users’ data beforehand. Nor should you have to. Your job is to ensure customers see the value in the software; not to struggle with importing data.

You could give them a spreadsheet template, but that would require them to spend time reformatting all their data. You could point them to the knowledgebase, but, again, that assumes that your end users will be willing to do that extra work.

In reality, your users are going to be in a hurry to get inside the new software and get to work. They’re not going to stop to deal with this. That’s the software’s job.

However, many data onboarding solutions don’t handle messy spreadsheets very well. Not only do they have a hard time recognizing what some of the data is (often because the data model doesn’t match their own), but then the application refuses to accept certain spreadsheet columns.

Even if it’s the end user’s fault for not properly organizing or labeling their data or teaching their team how to do so (or just not knowing what to do in the first place), who do you think they’re going to blame in the end when their data won’t import?

The Fix

Flatfile Concierge’s importer is AI-powered, which means that your software (and data importer) really can do the work for your end users.

Using advanced validation logic, the data importer can figure out what the data is and where it goes.

While Flatfile will automatically match columns and corresponding data to your software’s actual data fields, users get a chance to confirm that’s the case before allowing it into the system:

Before this happens, you can do a little work on the backend to ensure that Flatfile knows what to do with your users’ data:

  • Create target data models so Flatfile can navigate complex spreadsheet formats and datatypes your users will likely try to import.
  • Create a template with validation rules so Flatfile’s AI knows exactly how to map everything out.
  • Validate imported data against other databases to help the importer contextualize, validate and clean up the data over time.

Once you’ve done that upfront work, the rest is easy.

The bulk of the work will be done by Flatfile Concierge when it transforms imported data into something clean and useful. In fact, about 95% of imported columns will automatically map to your software thanks to Flatfile’s machine learning and fuzzy matching system.

The end user will have the opportunity to review the parts of their data that contain errors. If they find any, they can repair the errors inside Flatfile, rather than have to fix it in a spreadsheet and re-import.

Challenge #3: Getting And Tracking Data From Multiple Users

When there are a lot of cooks in the kitchen, there are a number of things that can go wrong.

Data can sometimes live on team members’ computers, or worse, sent over email, which can be a huge security concern for sensitive data. This can happen if users aren’t given access to the software platform or find the data importer too intimidating to use.

On the flip side of that, with the wrong data onboarding process, it could become like a free-for-all where people add whatever the heck they like to the company’s data. While the data does get imported, there’s no review framework so the company’s database is filled with errors and duplicate entries.

Your end users need to be able to maintain order, control and security when dealing with something as serious as company data — especially if you want your software to be usable.

The Fix

Flatfile Concierge has designed the data onboarding process to be a collaborative one.

As you can see, company admins can invite specific collaborators (i.e. customers) to add data to their workspaces. But this isn’t a blanket invitation to import data.

Admins have the ability to create an approval process. They get to:

  • Ask for specific data sets from team members.
  • Control which workspaces they’re allowed to import data to.
  • Review all data submissions before flowing the approved data into the platform.

Admins can also import data on the customer’s behalf. Flatfile Concierge ensures that data onboarding is never a dead-end for customers.

Not only does this ensure that the right data ends up in the software, but the controlled flow means the data will end up being cleaner and more accurate, too. All of this, while providing a seamless data onboarding experience for users.

Challenge #4: Data Security Is Always A Concern

When it comes to web and app development, user privacy and security are top priority. If our customers and visitors don’t trust that their information is safe from prying eyes (and isn’t being sold off to advertisers), they’re going to stop using our solutions in the first place.

The same thing happens with software — though it’s not just the company’s personal data they have to worry about securing.

Often, when companies import data into software (like the CRM example), they’re importing their customers’ private and sensitive data. Allow that to be compromised and you can kiss your software goodbye.

So, yes, the software itself needs to be secured. That’s a given. But so, too, does your data onboarding process. It’s a huge point of vulnerability if left unchecked.

The Fix

The first thing Flatfile Concierge does is to encourage users to move away from sharing sensitive data over email, FTP, and other unsecured platforms by providing a user-friendly data onboarding solution.

The second thing it does is provide an authenticated and compliant workspace for users to import, validate, and post their data to your software.

Here’s how Flatfile Concierge secures its workspaces:

  • Each collaborator enters the data importer through an authenticated invitation.
  • Data is encrypted in transit and stored in an encrypted Amazon S3 bucket.
  • The data onboarding platform is 100% GDPR compliant.
  • Flatfile is HIPAA and SOC2 compliant and can adjust for other compliance requirements as needed.

In addition, once data is successfully migrated into your application, it’s deleted from Flatfile. This way, you only have to worry about securing your data within your software and not on previous platforms it’s touched.

Wrapping Up

With an insufficient or error-prone data onboarding process, you, the software provider and its end users are going to spend too much time manually cleaning and validating spreadsheets. This won’t just happen during the initial user signup either. If the data importer isn’t up to the task, you’re all going to be throwing away a ton of time and resources every time data needs to be uploaded or transferred into the platform from existing customers.

Of course, this all assumes that your importer can even get user data into the software. (Sadly, this happens with too many custom-built solutions.)

Needless to say: Your data onboarding process must be flawless for your team and customers. It’s the only way to keep user churn rates low and user satisfaction high.

Data onboarding is a really complex process to handle. Save yourself the trouble in trying to develop your own data onboarding solution and the time trying to troubleshoot the problems with it. With an AI-powered data importer like Flatfile Concierge, everything’s taken care of for you.

Articles on Smashing Magazine — For Web Designers And Developers

09 Dec

How To Overcome Data Onboarding Challenges For Software Products

by TBSCategories: News

Companies willing to pay good money for a new piece of software are most likely not starting from scratch. They’re running an established business, with well-built and documented processes. So, they have tons of data to carry over.

As a result, the decision to bring a new app into the fold is not one they take lightly. Internal processes need to change. Getting the team to adopt the new solution can take time. Integrating it with existing systems and external tools can be a problem. Oh yeah, and there’s the matter of compliance to worry about, too.

This means there’s a lot of pressure on new software to provide a top-notch experience from the get-go. Fail to provide companies with a simple and intuitive way to onboard their data and you can expect high rates of customer churn as a result.

If you’re designing a product that needs data from customers in order to be of any value, here’s what you need to know about building out your data onboarding process.

How Data Onboarding Correlates With User Satisfaction

Business software is essentially just an empty box waiting to be filled with its users’ data. Without the ability to flawlessly onboard users’ data, the software essentially becomes useless.

Let’s look at what happens when you get the data onboarding process right.

End User Benefits

If you can nail the data onboarding piece, expect your end users to reap the following benefits:

  • They’ll be more confident in their decision.
    With complete and accurate data transferred into your software, users actually see how valuable it is soon after signing up. This leaves little room for second-guessing their decision, which leads to greater satisfaction overall with the product, and ultimately more money for your business.
  • You’ll get greater team buy-in.
    A positive data onboarding experience lets customers use your product faster, reducing the time needed for them to get value. So, really, data onboarding sets the stage for how your customers and their team will view the rest of your app.
  • They’ll experience more success with the software.
    Since users won’t have to stress about data formatting and cleanup or troubleshooting error-ridden import processes, they can get more out of the product and its features.

Software Developer Benefits

The software provider (you and/or your client) benefits, too:

  • Improve user satisfaction.
    Your end users don’t need to be technical wizards to figure out how to onboard data into your product. When you make light work of this, you reduce churn, attract more users and retain more loyal users over the long-term.
  • Spend less time on customer service.
    You can stop worrying about having to support a faulty data onboarding process as well as taking over tasks like data formatting and validation for your users. Instead, put your time and energy towards building better relationships with customers instead of putting out fires all the time.
    Kelly Abbott, Co-Founder and CTO of Tablecloth, can attest to this:

    “We have cut the amount of time we spend wrangling with files by 95%. We basically had all hands working to solve those problems at times.”

  • Have greater confidence in your product.
    When you have a data onboarding solution that’s flexible and powerful, you don’t have to restrict what data your users can or can’t import. It’s no longer a limitation.
    As Abbott explains:

    “It has made us more contemplative about the data we are asking clients for. We no longer have to avoid asking for data that may require too much time to fix. Flatfile eliminates that problem and has improved our willingness to experiment with different types of data we can incorporate into our analyses. The more time we spend tinkering with different data types, the more likely we are going to uncover the insight that produces additional value in the marketplace. That is indispensable for a startup like us.”

  • Save money.
    Although you’ll have to spend money on a third-party data onboarding solution, you’ll save your company the time and money otherwise spent trying to manage a custom-built data importer, onboarding process and client relationships. (Tablecloth, for instance, saved tens of thousands of dollars when they adopted Flatfile.)

The Challenges Of Data Onboarding For Software Products

Let’s have a look at the common challenges in data onboarding and how Flatfile Concierge removes them:

Challenge #1: There’s A Lot Of Data To Aggregate

When signing up for new business software, users probably expect to do a little work upfront, like filling in basic account information, configuring settings and adding users. The last thing you want to do is surprise them with a data importer that’s going to cause more work for them.

Let’s say, for example, you’ve built a CRM.

Unless the software targets startups and other new businesses, users are going to have a ton of external data to bring along. For instance:

  • Contact info for clients, prospects, vendors, partners and team members;
  • Existing customer data like account and sales history;
  • Prospect data like communication history;
  • Sales pipeline details;
  • Team and individual goals and metrics.

Unless your CRM directly integrates with every one of your users’ previous CRMs, how are they going to move this data over? Copy and paste? CSV templates?

Concierge allows companies to import ‘data model’ templates based on spreadsheet columns for easy setup. Customers would then import data that automatically matches this model with no code needed. (Image source: Flatfile)

Plus, you have to think about all of the other sources a CRM pulls info in from. Payment gateways. Spreadsheets that live on a sales team’s drive. Signed contracts that have been emailed or faxed to your company. There’s a lot of data coming from different places and people.

The Fix

There are a number of things Flatfile Concierge does to fix this problem.

For starters, it allows data to be imported from a variety of file types:

  • CSV,
  • TSV,
  • XLS,
  • XML,
  • And more.

With this kind of flexibility, your users won’t have to worry about transferring data to one specific file type and then cleaning up errors that occur during the transfer. Flatfile Concierge can handle various file types, of varying data types, and easily validate it all.

Another thing to think about is how your software is going to track and organize each imported file and its corresponding data.

What Flatfile allows your users to do is create collaborative workspaces to place data in. When a team member adds new data to the workspace, a record is captured containing the:

  • Date of upload,
  • File name,
  • User who submitted the data,
  • Number of rows added,
  • Version history,
  • Upload errors.

Flatfile Concierge animation demonstrating notifications for when spreadsheets are imported. (Image source: Flatfile)

This will keep things organized while also keeping everyone accountable to the data they contribute. And with this information readily available from a centralized dashboard, there’ll be no secret as to what’s been uploaded, by whom and when. Import errors can also be fixed collaboratively, without the need to re-upload spreadsheet data.

Challenge #2: Data Is Imported In A Variety Of States

When you give your software users the ability to transfer their data into your product, there’s not a lot you or the software team can do in terms of formatting or cleaning up end users’ data beforehand. Nor should you have to. Your job is to ensure customers see the value in the software; not to struggle with importing data.

You could give them a spreadsheet template, but that would require them to spend time reformatting all their data. You could point them to the knowledgebase, but, again, that assumes that your end users will be willing to do that extra work.

In reality, your users are going to be in a hurry to get inside the new software and get to work. They’re not going to stop to deal with this. That’s the software’s job.

However, many data onboarding solutions don’t handle messy spreadsheets very well. Not only do they have a hard time recognizing what some of the data is (often because the data model doesn’t match their own), but then the application refuses to accept certain spreadsheet columns.

Even if it’s the end user’s fault for not properly organizing or labeling their data or teaching their team how to do so (or just not knowing what to do in the first place), who do you think they’re going to blame in the end when their data won’t import?

The Fix

Flatfile Concierge’s importer is AI-powered, which means that your software (and data importer) really can do the work for your end users.

Using advanced validation logic, the data importer can figure out what the data is and where it goes.

While Flatfile will automatically match columns and corresponding data to your software’s actual data fields, users get a chance to confirm that’s the case before allowing it into the system:

Before this happens, you can do a little work on the backend to ensure that Flatfile knows what to do with your users’ data:

  • Create target data models so Flatfile can navigate complex spreadsheet formats and datatypes your users will likely try to import.
  • Create a template with validation rules so Flatfile’s AI knows exactly how to map everything out.
  • Validate imported data against other databases to help the importer contextualize, validate and clean up the data over time.

Once you’ve done that upfront work, the rest is easy.

The bulk of the work will be done by Flatfile Concierge when it transforms imported data into something clean and useful. In fact, about 95% of imported columns will automatically map to your software thanks to Flatfile’s machine learning and fuzzy matching system.

The end user will have the opportunity to review the parts of their data that contain errors. If they find any, they can repair the errors inside Flatfile, rather than have to fix it in a spreadsheet and re-import.

Challenge #3: Getting And Tracking Data From Multiple Users

When there are a lot of cooks in the kitchen, there are a number of things that can go wrong.

Data can sometimes live on team members’ computers, or worse, sent over email, which can be a huge security concern for sensitive data. This can happen if users aren’t given access to the software platform or find the data importer too intimidating to use.

On the flip side of that, with the wrong data onboarding process, it could become like a free-for-all where people add whatever the heck they like to the company’s data. While the data does get imported, there’s no review framework so the company’s database is filled with errors and duplicate entries.

Your end users need to be able to maintain order, control and security when dealing with something as serious as company data — especially if you want your software to be usable.

The Fix

Flatfile Concierge has designed the data onboarding process to be a collaborative one.

As you can see, company admins can invite specific collaborators (i.e. customers) to add data to their workspaces. But this isn’t a blanket invitation to import data.

Admins have the ability to create an approval process. They get to:

  • Ask for specific data sets from team members.
  • Control which workspaces they’re allowed to import data to.
  • Review all data submissions before flowing the approved data into the platform.

Admins can also import data on the customer’s behalf. Flatfile Concierge ensures that data onboarding is never a dead-end for customers.

Not only does this ensure that the right data ends up in the software, but the controlled flow means the data will end up being cleaner and more accurate, too. All of this, while providing a seamless data onboarding experience for users.

Challenge #4: Data Security Is Always A Concern

When it comes to web and app development, user privacy and security are top priority. If our customers and visitors don’t trust that their information is safe from prying eyes (and isn’t being sold off to advertisers), they’re going to stop using our solutions in the first place.

The same thing happens with software — though it’s not just the company’s personal data they have to worry about securing.

Often, when companies import data into software (like the CRM example), they’re importing their customers’ private and sensitive data. Allow that to be compromised and you can kiss your software goodbye.

So, yes, the software itself needs to be secured. That’s a given. But so, too, does your data onboarding process. It’s a huge point of vulnerability if left unchecked.

The Fix

The first thing Flatfile Concierge does is to encourage users to move away from sharing sensitive data over email, FTP, and other unsecured platforms by providing a user-friendly data onboarding solution.

The second thing it does is provide an authenticated and compliant workspace for users to import, validate, and post their data to your software.

Here’s how Flatfile Concierge secures its workspaces:

  • Each collaborator enters the data importer through an authenticated invitation.
  • Data is encrypted in transit and stored in an encrypted Amazon S3 bucket.
  • The data onboarding platform is 100% GDPR compliant.
  • Flatfile is HIPAA and SOC2 compliant and can adjust for other compliance requirements as needed.

In addition, once data is successfully migrated into your application, it’s deleted from Flatfile. This way, you only have to worry about securing your data within your software and not on previous platforms it’s touched.

Wrapping Up

With an insufficient or error-prone data onboarding process, you, the software provider and its end users are going to spend too much time manually cleaning and validating spreadsheets. This won’t just happen during the initial user signup either. If the data importer isn’t up to the task, you’re all going to be throwing away a ton of time and resources every time data needs to be uploaded or transferred into the platform from existing customers.

Of course, this all assumes that your importer can even get user data into the software. (Sadly, this happens with too many custom-built solutions.)

Needless to say: Your data onboarding process must be flawless for your team and customers. It’s the only way to keep user churn rates low and user satisfaction high.

Data onboarding is a really complex process to handle. Save yourself the trouble in trying to develop your own data onboarding solution and the time trying to troubleshoot the problems with it. With an AI-powered data importer like Flatfile Concierge, everything’s taken care of for you.

Articles on Smashing Magazine — For Web Designers And Developers

09 Dec

How To Overcome Data Onboarding Challenges For Software Products

by TBSCategories: News

Companies willing to pay good money for a new piece of software are most likely not starting from scratch. They’re running an established business, with well-built and documented processes. So, they have tons of data to carry over.

As a result, the decision to bring a new app into the fold is not one they take lightly. Internal processes need to change. Getting the team to adopt the new solution can take time. Integrating it with existing systems and external tools can be a problem. Oh yeah, and there’s the matter of compliance to worry about, too.

This means there’s a lot of pressure on new software to provide a top-notch experience from the get-go. Fail to provide companies with a simple and intuitive way to onboard their data and you can expect high rates of customer churn as a result.

If you’re designing a product that needs data from customers in order to be of any value, here’s what you need to know about building out your data onboarding process.

How Data Onboarding Correlates With User Satisfaction

Business software is essentially just an empty box waiting to be filled with its users’ data. Without the ability to flawlessly onboard users’ data, the software essentially becomes useless.

Let’s look at what happens when you get the data onboarding process right.

End User Benefits

If you can nail the data onboarding piece, expect your end users to reap the following benefits:

  • They’ll be more confident in their decision.
    With complete and accurate data transferred into your software, users actually see how valuable it is soon after signing up. This leaves little room for second-guessing their decision, which leads to greater satisfaction overall with the product, and ultimately more money for your business.
  • You’ll get greater team buy-in.
    A positive data onboarding experience lets customers use your product faster, reducing the time needed for them to get value. So, really, data onboarding sets the stage for how your customers and their team will view the rest of your app.
  • They’ll experience more success with the software.
    Since users won’t have to stress about data formatting and cleanup or troubleshooting error-ridden import processes, they can get more out of the product and its features.

Software Developer Benefits

The software provider (you and/or your client) benefits, too:

  • Improve user satisfaction.
    Your end users don’t need to be technical wizards to figure out how to onboard data into your product. When you make light work of this, you reduce churn, attract more users and retain more loyal users over the long-term.
  • Spend less time on customer service.
    You can stop worrying about having to support a faulty data onboarding process as well as taking over tasks like data formatting and validation for your users. Instead, put your time and energy towards building better relationships with customers instead of putting out fires all the time.
    Kelly Abbott, Co-Founder and CTO of Tablecloth, can attest to this:

    “We have cut the amount of time we spend wrangling with files by 95%. We basically had all hands working to solve those problems at times.”

  • Have greater confidence in your product.
    When you have a data onboarding solution that’s flexible and powerful, you don’t have to restrict what data your users can or can’t import. It’s no longer a limitation.
    As Abbott explains:

    “It has made us more contemplative about the data we are asking clients for. We no longer have to avoid asking for data that may require too much time to fix. Flatfile eliminates that problem and has improved our willingness to experiment with different types of data we can incorporate into our analyses. The more time we spend tinkering with different data types, the more likely we are going to uncover the insight that produces additional value in the marketplace. That is indispensable for a startup like us.”

  • Save money.
    Although you’ll have to spend money on a third-party data onboarding solution, you’ll save your company the time and money otherwise spent trying to manage a custom-built data importer, onboarding process and client relationships. (Tablecloth, for instance, saved tens of thousands of dollars when they adopted Flatfile.)

The Challenges Of Data Onboarding For Software Products

Let’s have a look at the common challenges in data onboarding and how Flatfile Concierge removes them:

Challenge #1: There’s A Lot Of Data To Aggregate

When signing up for new business software, users probably expect to do a little work upfront, like filling in basic account information, configuring settings and adding users. The last thing you want to do is surprise them with a data importer that’s going to cause more work for them.

Let’s say, for example, you’ve built a CRM.

Unless the software targets startups and other new businesses, users are going to have a ton of external data to bring along. For instance:

  • Contact info for clients, prospects, vendors, partners and team members;
  • Existing customer data like account and sales history;
  • Prospect data like communication history;
  • Sales pipeline details;
  • Team and individual goals and metrics.

Unless your CRM directly integrates with every one of your users’ previous CRMs, how are they going to move this data over? Copy and paste? CSV templates?

Concierge allows companies to import ‘data model’ templates based on spreadsheet columns for easy setup. Customers would then import data that automatically matches this model with no code needed. (Image source: Flatfile)

Plus, you have to think about all of the other sources a CRM pulls info in from. Payment gateways. Spreadsheets that live on a sales team’s drive. Signed contracts that have been emailed or faxed to your company. There’s a lot of data coming from different places and people.

The Fix

There are a number of things Flatfile Concierge does to fix this problem.

For starters, it allows data to be imported from a variety of file types:

  • CSV,
  • TSV,
  • XLS,
  • XML,
  • And more.

With this kind of flexibility, your users won’t have to worry about transferring data to one specific file type and then cleaning up errors that occur during the transfer. Flatfile Concierge can handle various file types, of varying data types, and easily validate it all.

Another thing to think about is how your software is going to track and organize each imported file and its corresponding data.

What Flatfile allows your users to do is create collaborative workspaces to place data in. When a team member adds new data to the workspace, a record is captured containing the:

  • Date of upload,
  • File name,
  • User who submitted the data,
  • Number of rows added,
  • Version history,
  • Upload errors.

Flatfile Concierge animation demonstrating notifications for when spreadsheets are imported. (Image source: Flatfile)

This will keep things organized while also keeping everyone accountable to the data they contribute. And with this information readily available from a centralized dashboard, there’ll be no secret as to what’s been uploaded, by whom and when. Import errors can also be fixed collaboratively, without the need to re-upload spreadsheet data.

Challenge #2: Data Is Imported In A Variety Of States

When you give your software users the ability to transfer their data into your product, there’s not a lot you or the software team can do in terms of formatting or cleaning up end users’ data beforehand. Nor should you have to. Your job is to ensure customers see the value in the software; not to struggle with importing data.

You could give them a spreadsheet template, but that would require them to spend time reformatting all their data. You could point them to the knowledgebase, but, again, that assumes that your end users will be willing to do that extra work.

In reality, your users are going to be in a hurry to get inside the new software and get to work. They’re not going to stop to deal with this. That’s the software’s job.

However, many data onboarding solutions don’t handle messy spreadsheets very well. Not only do they have a hard time recognizing what some of the data is (often because the data model doesn’t match their own), but then the application refuses to accept certain spreadsheet columns.

Even if it’s the end user’s fault for not properly organizing or labeling their data or teaching their team how to do so (or just not knowing what to do in the first place), who do you think they’re going to blame in the end when their data won’t import?

The Fix

Flatfile Concierge’s importer is AI-powered, which means that your software (and data importer) really can do the work for your end users.

Using advanced validation logic, the data importer can figure out what the data is and where it goes.

While Flatfile will automatically match columns and corresponding data to your software’s actual data fields, users get a chance to confirm that’s the case before allowing it into the system:

Before this happens, you can do a little work on the backend to ensure that Flatfile knows what to do with your users’ data:

  • Create target data models so Flatfile can navigate complex spreadsheet formats and datatypes your users will likely try to import.
  • Create a template with validation rules so Flatfile’s AI knows exactly how to map everything out.
  • Validate imported data against other databases to help the importer contextualize, validate and clean up the data over time.

Once you’ve done that upfront work, the rest is easy.

The bulk of the work will be done by Flatfile Concierge when it transforms imported data into something clean and useful. In fact, about 95% of imported columns will automatically map to your software thanks to Flatfile’s machine learning and fuzzy matching system.

The end user will have the opportunity to review the parts of their data that contain errors. If they find any, they can repair the errors inside Flatfile, rather than have to fix it in a spreadsheet and re-import.

Challenge #3: Getting And Tracking Data From Multiple Users

When there are a lot of cooks in the kitchen, there are a number of things that can go wrong.

Data can sometimes live on team members’ computers, or worse, sent over email, which can be a huge security concern for sensitive data. This can happen if users aren’t given access to the software platform or find the data importer too intimidating to use.

On the flip side of that, with the wrong data onboarding process, it could become like a free-for-all where people add whatever the heck they like to the company’s data. While the data does get imported, there’s no review framework so the company’s database is filled with errors and duplicate entries.

Your end users need to be able to maintain order, control and security when dealing with something as serious as company data — especially if you want your software to be usable.

The Fix

Flatfile Concierge has designed the data onboarding process to be a collaborative one.

As you can see, company admins can invite specific collaborators (i.e. customers) to add data to their workspaces. But this isn’t a blanket invitation to import data.

Admins have the ability to create an approval process. They get to:

  • Ask for specific data sets from team members.
  • Control which workspaces they’re allowed to import data to.
  • Review all data submissions before flowing the approved data into the platform.

Admins can also import data on the customer’s behalf. Flatfile Concierge ensures that data onboarding is never a dead-end for customers.

Not only does this ensure that the right data ends up in the software, but the controlled flow means the data will end up being cleaner and more accurate, too. All of this, while providing a seamless data onboarding experience for users.

Challenge #4: Data Security Is Always A Concern

When it comes to web and app development, user privacy and security are top priority. If our customers and visitors don’t trust that their information is safe from prying eyes (and isn’t being sold off to advertisers), they’re going to stop using our solutions in the first place.

The same thing happens with software — though it’s not just the company’s personal data they have to worry about securing.

Often, when companies import data into software (like the CRM example), they’re importing their customers’ private and sensitive data. Allow that to be compromised and you can kiss your software goodbye.

So, yes, the software itself needs to be secured. That’s a given. But so, too, does your data onboarding process. It’s a huge point of vulnerability if left unchecked.

The Fix

The first thing Flatfile Concierge does is to encourage users to move away from sharing sensitive data over email, FTP, and other unsecured platforms by providing a user-friendly data onboarding solution.

The second thing it does is provide an authenticated and compliant workspace for users to import, validate, and post their data to your software.

Here’s how Flatfile Concierge secures its workspaces:

  • Each collaborator enters the data importer through an authenticated invitation.
  • Data is encrypted in transit and stored in an encrypted Amazon S3 bucket.
  • The data onboarding platform is 100% GDPR compliant.
  • Flatfile is HIPAA and SOC2 compliant and can adjust for other compliance requirements as needed.

In addition, once data is successfully migrated into your application, it’s deleted from Flatfile. This way, you only have to worry about securing your data within your software and not on previous platforms it’s touched.

Wrapping Up

With an insufficient or error-prone data onboarding process, you, the software provider and its end users are going to spend too much time manually cleaning and validating spreadsheets. This won’t just happen during the initial user signup either. If the data importer isn’t up to the task, you’re all going to be throwing away a ton of time and resources every time data needs to be uploaded or transferred into the platform from existing customers.

Of course, this all assumes that your importer can even get user data into the software. (Sadly, this happens with too many custom-built solutions.)

Needless to say: Your data onboarding process must be flawless for your team and customers. It’s the only way to keep user churn rates low and user satisfaction high.

Data onboarding is a really complex process to handle. Save yourself the trouble in trying to develop your own data onboarding solution and the time trying to troubleshoot the problems with it. With an AI-powered data importer like Flatfile Concierge, everything’s taken care of for you.

Articles on Smashing Magazine — For Web Designers And Developers

08 Dec

How To Overcome Data Onboarding Challenges For Software Products

by TBSCategories: News

Companies willing to pay good money for a new piece of software are most likely not starting from scratch. They’re running an established business, with well-built and documented processes. So, they have tons of data to carry over.

As a result, the decision to bring a new app into the fold is not one they take lightly. Internal processes need to change. Getting the team to adopt the new solution can take time. Integrating it with existing systems and external tools can be a problem. Oh yeah, and there’s the matter of compliance to worry about, too.

This means there’s a lot of pressure on new software to provide a top-notch experience from the get-go. Fail to provide companies with a simple and intuitive way to onboard their data and you can expect high rates of customer churn as a result.

If you’re designing a product that needs data from customers in order to be of any value, here’s what you need to know about building out your data onboarding process.

How Data Onboarding Correlates With User Satisfaction

Business software is essentially just an empty box waiting to be filled with its users’ data. Without the ability to flawlessly onboard users’ data, the software essentially becomes useless.

Let’s look at what happens when you get the data onboarding process right.

End User Benefits

If you can nail the data onboarding piece, expect your end users to reap the following benefits:

  • They’ll be more confident in their decision.
    With complete and accurate data transferred into your software, users actually see how valuable it is soon after signing up. This leaves little room for second-guessing their decision, which leads to greater satisfaction overall with the product, and ultimately more money for your business.
  • You’ll get greater team buy-in.
    A positive data onboarding experience lets customers use your product faster, reducing the time needed for them to get value. So, really, data onboarding sets the stage for how your customers and their team will view the rest of your app.
  • They’ll experience more success with the software.
    Since users won’t have to stress about data formatting and cleanup or troubleshooting error-ridden import processes, they can get more out of the product and its features.

Software Developer Benefits

The software provider (you and/or your client) benefits, too:

  • Improve user satisfaction.
    Your end users don’t need to be technical wizards to figure out how to onboard data into your product. When you make light work of this, you reduce churn, attract more users and retain more loyal users over the long-term.
  • Spend less time on customer service.
    You can stop worrying about having to support a faulty data onboarding process as well as taking over tasks like data formatting and validation for your users. Instead, put your time and energy towards building better relationships with customers instead of putting out fires all the time.
    Kelly Abbott, Co-Founder and CTO of Tablecloth, can attest to this:

    “We have cut the amount of time we spend wrangling with files by 95%. We basically had all hands working to solve those problems at times.”

  • Have greater confidence in your product.
    When you have a data onboarding solution that’s flexible and powerful, you don’t have to restrict what data your users can or can’t import. It’s no longer a limitation.
    As Abbott explains:

    “It has made us more contemplative about the data we are asking clients for. We no longer have to avoid asking for data that may require too much time to fix. Flatfile eliminates that problem and has improved our willingness to experiment with different types of data we can incorporate into our analyses. The more time we spend tinkering with different data types, the more likely we are going to uncover the insight that produces additional value in the marketplace. That is indispensable for a startup like us.”

  • Save money.
    Although you’ll have to spend money on a third-party data onboarding solution, you’ll save your company the time and money otherwise spent trying to manage a custom-built data importer, onboarding process and client relationships. (Tablecloth, for instance, saved tens of thousands of dollars when they adopted Flatfile.)

The Challenges Of Data Onboarding For Software Products

Let’s have a look at the common challenges in data onboarding and how Flatfile Concierge removes them:

Challenge #1: There’s A Lot Of Data To Aggregate

When signing up for new business software, users probably expect to do a little work upfront, like filling in basic account information, configuring settings and adding users. The last thing you want to do is surprise them with a data importer that’s going to cause more work for them.

Let’s say, for example, you’ve built a CRM.

Unless the software targets startups and other new businesses, users are going to have a ton of external data to bring along. For instance:

  • Contact info for clients, prospects, vendors, partners and team members;
  • Existing customer data like account and sales history;
  • Prospect data like communication history;
  • Sales pipeline details;
  • Team and individual goals and metrics.

Unless your CRM directly integrates with every one of your users’ previous CRMs, how are they going to move this data over? Copy and paste? CSV templates?

Concierge allows companies to import ‘data model’ templates based on spreadsheet columns for easy setup. Customers would then import data that automatically matches this model with no code needed. (Image source: Flatfile)

Plus, you have to think about all of the other sources a CRM pulls info in from. Payment gateways. Spreadsheets that live on a sales team’s drive. Signed contracts that have been emailed or faxed to your company. There’s a lot of data coming from different places and people.

The Fix

There are a number of things Flatfile Concierge does to fix this problem.

For starters, it allows data to be imported from a variety of file types:

  • CSV,
  • TSV,
  • XLS,
  • XML,
  • And more.

With this kind of flexibility, your users won’t have to worry about transferring data to one specific file type and then cleaning up errors that occur during the transfer. Flatfile Concierge can handle various file types, of varying data types, and easily validate it all.

Another thing to think about is how your software is going to track and organize each imported file and its corresponding data.

What Flatfile allows your users to do is create collaborative workspaces to place data in. When a team member adds new data to the workspace, a record is captured containing the:

  • Date of upload,
  • File name,
  • User who submitted the data,
  • Number of rows added,
  • Version history,
  • Upload errors.

Flatfile Concierge animation demonstrating notifications for when spreadsheets are imported. (Image source: Flatfile)

This will keep things organized while also keeping everyone accountable to the data they contribute. And with this information readily available from a centralized dashboard, there’ll be no secret as to what’s been uploaded, by whom and when. Import errors can also be fixed collaboratively, without the need to re-upload spreadsheet data.

Challenge #2: Data Is Imported In A Variety Of States

When you give your software users the ability to transfer their data into your product, there’s not a lot you or the software team can do in terms of formatting or cleaning up end users’ data beforehand. Nor should you have to. Your job is to ensure customers see the value in the software; not to struggle with importing data.

You could give them a spreadsheet template, but that would require them to spend time reformatting all their data. You could point them to the knowledgebase, but, again, that assumes that your end users will be willing to do that extra work.

In reality, your users are going to be in a hurry to get inside the new software and get to work. They’re not going to stop to deal with this. That’s the software’s job.

However, many data onboarding solutions don’t handle messy spreadsheets very well. Not only do they have a hard time recognizing what some of the data is (often because the data model doesn’t match their own), but then the application refuses to accept certain spreadsheet columns.

Even if it’s the end user’s fault for not properly organizing or labeling their data or teaching their team how to do so (or just not knowing what to do in the first place), who do you think they’re going to blame in the end when their data won’t import?

The Fix

Flatfile Concierge’s importer is AI-powered, which means that your software (and data importer) really can do the work for your end users.

Using advanced validation logic, the data importer can figure out what the data is and where it goes.

While Flatfile will automatically match columns and corresponding data to your software’s actual data fields, users get a chance to confirm that’s the case before allowing it into the system:

Before this happens, you can do a little work on the backend to ensure that Flatfile knows what to do with your users’ data:

  • Create target data models so Flatfile can navigate complex spreadsheet formats and datatypes your users will likely try to import.
  • Create a template with validation rules so Flatfile’s AI knows exactly how to map everything out.
  • Validate imported data against other databases to help the importer contextualize, validate and clean up the data over time.

Once you’ve done that upfront work, the rest is easy.

The bulk of the work will be done by Flatfile Concierge when it transforms imported data into something clean and useful. In fact, about 95% of imported columns will automatically map to your software thanks to Flatfile’s machine learning and fuzzy matching system.

The end user will have the opportunity to review the parts of their data that contain errors. If they find any, they can repair the errors inside Flatfile, rather than have to fix it in a spreadsheet and re-import.

Challenge #3: Getting And Tracking Data From Multiple Users

When there are a lot of cooks in the kitchen, there are a number of things that can go wrong.

Data can sometimes live on team members’ computers, or worse, sent over email, which can be a huge security concern for sensitive data. This can happen if users aren’t given access to the software platform or find the data importer too intimidating to use.

On the flip side of that, with the wrong data onboarding process, it could become like a free-for-all where people add whatever the heck they like to the company’s data. While the data does get imported, there’s no review framework so the company’s database is filled with errors and duplicate entries.

Your end users need to be able to maintain order, control and security when dealing with something as serious as company data — especially if you want your software to be usable.

The Fix

Flatfile Concierge has designed the data onboarding process to be a collaborative one.

As you can see, company admins can invite specific collaborators (i.e. customers) to add data to their workspaces. But this isn’t a blanket invitation to import data.

Admins have the ability to create an approval process. They get to:

  • Ask for specific data sets from team members.
  • Control which workspaces they’re allowed to import data to.
  • Review all data submissions before flowing the approved data into the platform.

Admins can also import data on the customer’s behalf. Flatfile Concierge ensures that data onboarding is never a dead-end for customers.

Not only does this ensure that the right data ends up in the software, but the controlled flow means the data will end up being cleaner and more accurate, too. All of this, while providing a seamless data onboarding experience for users.

Challenge #4: Data Security Is Always A Concern

When it comes to web and app development, user privacy and security are top priority. If our customers and visitors don’t trust that their information is safe from prying eyes (and isn’t being sold off to advertisers), they’re going to stop using our solutions in the first place.

The same thing happens with software — though it’s not just the company’s personal data they have to worry about securing.

Often, when companies import data into software (like the CRM example), they’re importing their customers’ private and sensitive data. Allow that to be compromised and you can kiss your software goodbye.

So, yes, the software itself needs to be secured. That’s a given. But so, too, does your data onboarding process. It’s a huge point of vulnerability if left unchecked.

The Fix

The first thing Flatfile Concierge does is to encourage users to move away from sharing sensitive data over email, FTP, and other unsecured platforms by providing a user-friendly data onboarding solution.

The second thing it does is provide an authenticated and compliant workspace for users to import, validate, and post their data to your software.

Here’s how Flatfile Concierge secures its workspaces:

  • Each collaborator enters the data importer through an authenticated invitation.
  • Data is encrypted in transit and stored in an encrypted Amazon S3 bucket.
  • The data onboarding platform is 100% GDPR compliant.
  • Flatfile is HIPAA and SOC2 compliant and can adjust for other compliance requirements as needed.

In addition, once data is successfully migrated into your application, it’s deleted from Flatfile. This way, you only have to worry about securing your data within your software and not on previous platforms it’s touched.

Wrapping Up

With an insufficient or error-prone data onboarding process, you, the software provider and its end users are going to spend too much time manually cleaning and validating spreadsheets. This won’t just happen during the initial user signup either. If the data importer isn’t up to the task, you’re all going to be throwing away a ton of time and resources every time data needs to be uploaded or transferred into the platform from existing customers.

Of course, this all assumes that your importer can even get user data into the software. (Sadly, this happens with too many custom-built solutions.)

Needless to say: Your data onboarding process must be flawless for your team and customers. It’s the only way to keep user churn rates low and user satisfaction high.

Data onboarding is a really complex process to handle. Save yourself the trouble in trying to develop your own data onboarding solution and the time trying to troubleshoot the problems with it. With an AI-powered data importer like Flatfile Concierge, everything’s taken care of for you.

Articles on Smashing Magazine — For Web Designers And Developers

08 Dec

How To Overcome Data Onboarding Challenges For Software Products

by TBSCategories: News

Companies willing to pay good money for a new piece of software are most likely not starting from scratch. They’re running an established business, with well-built and documented processes. So, they have tons of data to carry over.

As a result, the decision to bring a new app into the fold is not one they take lightly. Internal processes need to change. Getting the team to adopt the new solution can take time. Integrating it with existing systems and external tools can be a problem. Oh yeah, and there’s the matter of compliance to worry about, too.

This means there’s a lot of pressure on new software to provide a top-notch experience from the get-go. Fail to provide companies with a simple and intuitive way to onboard their data and you can expect high rates of customer churn as a result.

If you’re designing a product that needs data from customers in order to be of any value, here’s what you need to know about building out your data onboarding process.

How Data Onboarding Correlates With User Satisfaction

Business software is essentially just an empty box waiting to be filled with its users’ data. Without the ability to flawlessly onboard users’ data, the software essentially becomes useless.

Let’s look at what happens when you get the data onboarding process right.

End User Benefits

If you can nail the data onboarding piece, expect your end users to reap the following benefits:

  • They’ll be more confident in their decision.
    With complete and accurate data transferred into your software, users actually see how valuable it is soon after signing up. This leaves little room for second-guessing their decision, which leads to greater satisfaction overall with the product, and ultimately more money for your business.
  • You’ll get greater team buy-in.
    A positive data onboarding experience lets customers use your product faster, reducing the time needed for them to get value. So, really, data onboarding sets the stage for how your customers and their team will view the rest of your app.
  • They’ll experience more success with the software.
    Since users won’t have to stress about data formatting and cleanup or troubleshooting error-ridden import processes, they can get more out of the product and its features.

Software Developer Benefits

The software provider (you and/or your client) benefits, too:

  • Improve user satisfaction.
    Your end users don’t need to be technical wizards to figure out how to onboard data into your product. When you make light work of this, you reduce churn, attract more users and retain more loyal users over the long-term.
  • Spend less time on customer service.
    You can stop worrying about having to support a faulty data onboarding process as well as taking over tasks like data formatting and validation for your users. Instead, put your time and energy towards building better relationships with customers instead of putting out fires all the time.
    Kelly Abbott, Co-Founder and CTO of Tablecloth, can attest to this:

    “We have cut the amount of time we spend wrangling with files by 95%. We basically had all hands working to solve those problems at times.”

  • Have greater confidence in your product.
    When you have a data onboarding solution that’s flexible and powerful, you don’t have to restrict what data your users can or can’t import. It’s no longer a limitation.
    As Abbott explains:

    “It has made us more contemplative about the data we are asking clients for. We no longer have to avoid asking for data that may require too much time to fix. Flatfile eliminates that problem and has improved our willingness to experiment with different types of data we can incorporate into our analyses. The more time we spend tinkering with different data types, the more likely we are going to uncover the insight that produces additional value in the marketplace. That is indispensable for a startup like us.”

  • Save money.
    Although you’ll have to spend money on a third-party data onboarding solution, you’ll save your company the time and money otherwise spent trying to manage a custom-built data importer, onboarding process and client relationships. (Tablecloth, for instance, saved tens of thousands of dollars when they adopted Flatfile.)

The Challenges Of Data Onboarding For Software Products

Let’s have a look at the common challenges in data onboarding and how Flatfile Concierge removes them:

Challenge #1: There’s A Lot Of Data To Aggregate

When signing up for new business software, users probably expect to do a little work upfront, like filling in basic account information, configuring settings and adding users. The last thing you want to do is surprise them with a data importer that’s going to cause more work for them.

Let’s say, for example, you’ve built a CRM.

Unless the software targets startups and other new businesses, users are going to have a ton of external data to bring along. For instance:

  • Contact info for clients, prospects, vendors, partners and team members;
  • Existing customer data like account and sales history;
  • Prospect data like communication history;
  • Sales pipeline details;
  • Team and individual goals and metrics.

Unless your CRM directly integrates with every one of your users’ previous CRMs, how are they going to move this data over? Copy and paste? CSV templates?

Concierge allows companies to import ‘data model’ templates based on spreadsheet columns for easy setup. Customers would then import data that automatically matches this model with no code needed. (Image source: Flatfile)

Plus, you have to think about all of the other sources a CRM pulls info in from. Payment gateways. Spreadsheets that live on a sales team’s drive. Signed contracts that have been emailed or faxed to your company. There’s a lot of data coming from different places and people.

The Fix

There are a number of things Flatfile Concierge does to fix this problem.

For starters, it allows data to be imported from a variety of file types:

  • CSV,
  • TSV,
  • XLS,
  • XML,
  • And more.

With this kind of flexibility, your users won’t have to worry about transferring data to one specific file type and then cleaning up errors that occur during the transfer. Flatfile Concierge can handle various file types, of varying data types, and easily validate it all.

Another thing to think about is how your software is going to track and organize each imported file and its corresponding data.

What Flatfile allows your users to do is create collaborative workspaces to place data in. When a team member adds new data to the workspace, a record is captured containing the:

  • Date of upload,
  • File name,
  • User who submitted the data,
  • Number of rows added,
  • Version history,
  • Upload errors.

Flatfile Concierge animation demonstrating notifications for when spreadsheets are imported. (Image source: Flatfile)

This will keep things organized while also keeping everyone accountable to the data they contribute. And with this information readily available from a centralized dashboard, there’ll be no secret as to what’s been uploaded, by whom and when. Import errors can also be fixed collaboratively, without the need to re-upload spreadsheet data.

Challenge #2: Data Is Imported In A Variety Of States

When you give your software users the ability to transfer their data into your product, there’s not a lot you or the software team can do in terms of formatting or cleaning up end users’ data beforehand. Nor should you have to. Your job is to ensure customers see the value in the software; not to struggle with importing data.

You could give them a spreadsheet template, but that would require them to spend time reformatting all their data. You could point them to the knowledgebase, but, again, that assumes that your end users will be willing to do that extra work.

In reality, your users are going to be in a hurry to get inside the new software and get to work. They’re not going to stop to deal with this. That’s the software’s job.

However, many data onboarding solutions don’t handle messy spreadsheets very well. Not only do they have a hard time recognizing what some of the data is (often because the data model doesn’t match their own), but then the application refuses to accept certain spreadsheet columns.

Even if it’s the end user’s fault for not properly organizing or labeling their data or teaching their team how to do so (or just not knowing what to do in the first place), who do you think they’re going to blame in the end when their data won’t import?

The Fix

Flatfile Concierge’s importer is AI-powered, which means that your software (and data importer) really can do the work for your end users.

Using advanced validation logic, the data importer can figure out what the data is and where it goes.

While Flatfile will automatically match columns and corresponding data to your software’s actual data fields, users get a chance to confirm that’s the case before allowing it into the system:

Before this happens, you can do a little work on the backend to ensure that Flatfile knows what to do with your users’ data:

  • Create target data models so Flatfile can navigate complex spreadsheet formats and datatypes your users will likely try to import.
  • Create a template with validation rules so Flatfile’s AI knows exactly how to map everything out.
  • Validate imported data against other databases to help the importer contextualize, validate and clean up the data over time.

Once you’ve done that upfront work, the rest is easy.

The bulk of the work will be done by Flatfile Concierge when it transforms imported data into something clean and useful. In fact, about 95% of imported columns will automatically map to your software thanks to Flatfile’s machine learning and fuzzy matching system.

The end user will have the opportunity to review the parts of their data that contain errors. If they find any, they can repair the errors inside Flatfile, rather than have to fix it in a spreadsheet and re-import.

Challenge #3: Getting And Tracking Data From Multiple Users

When there are a lot of cooks in the kitchen, there are a number of things that can go wrong.

Data can sometimes live on team members’ computers, or worse, sent over email, which can be a huge security concern for sensitive data. This can happen if users aren’t given access to the software platform or find the data importer too intimidating to use.

On the flip side of that, with the wrong data onboarding process, it could become like a free-for-all where people add whatever the heck they like to the company’s data. While the data does get imported, there’s no review framework so the company’s database is filled with errors and duplicate entries.

Your end users need to be able to maintain order, control and security when dealing with something as serious as company data — especially if you want your software to be usable.

The Fix

Flatfile Concierge has designed the data onboarding process to be a collaborative one.

As you can see, company admins can invite specific collaborators (i.e. customers) to add data to their workspaces. But this isn’t a blanket invitation to import data.

Admins have the ability to create an approval process. They get to:

  • Ask for specific data sets from team members.
  • Control which workspaces they’re allowed to import data to.
  • Review all data submissions before flowing the approved data into the platform.

Admins can also import data on the customer’s behalf. Flatfile Concierge ensures that data onboarding is never a dead-end for customers.

Not only does this ensure that the right data ends up in the software, but the controlled flow means the data will end up being cleaner and more accurate, too. All of this, while providing a seamless data onboarding experience for users.

Challenge #4: Data Security Is Always A Concern

When it comes to web and app development, user privacy and security are top priority. If our customers and visitors don’t trust that their information is safe from prying eyes (and isn’t being sold off to advertisers), they’re going to stop using our solutions in the first place.

The same thing happens with software — though it’s not just the company’s personal data they have to worry about securing.

Often, when companies import data into software (like the CRM example), they’re importing their customers’ private and sensitive data. Allow that to be compromised and you can kiss your software goodbye.

So, yes, the software itself needs to be secured. That’s a given. But so, too, does your data onboarding process. It’s a huge point of vulnerability if left unchecked.

The Fix

The first thing Flatfile Concierge does is to encourage users to move away from sharing sensitive data over email, FTP, and other unsecured platforms by providing a user-friendly data onboarding solution.

The second thing it does is provide an authenticated and compliant workspace for users to import, validate, and post their data to your software.

Here’s how Flatfile Concierge secures its workspaces:

  • Each collaborator enters the data importer through an authenticated invitation.
  • Data is encrypted in transit and stored in an encrypted Amazon S3 bucket.
  • The data onboarding platform is 100% GDPR compliant.
  • Flatfile is HIPAA and SOC2 compliant and can adjust for other compliance requirements as needed.

In addition, once data is successfully migrated into your application, it’s deleted from Flatfile. This way, you only have to worry about securing your data within your software and not on previous platforms it’s touched.

Wrapping Up

With an insufficient or error-prone data onboarding process, you, the software provider and its end users are going to spend too much time manually cleaning and validating spreadsheets. This won’t just happen during the initial user signup either. If the data importer isn’t up to the task, you’re all going to be throwing away a ton of time and resources every time data needs to be uploaded or transferred into the platform from existing customers.

Of course, this all assumes that your importer can even get user data into the software. (Sadly, this happens with too many custom-built solutions.)

Needless to say: Your data onboarding process must be flawless for your team and customers. It’s the only way to keep user churn rates low and user satisfaction high.

Data onboarding is a really complex process to handle. Save yourself the trouble in trying to develop your own data onboarding solution and the time trying to troubleshoot the problems with it. With an AI-powered data importer like Flatfile Concierge, everything’s taken care of for you.

Articles on Smashing Magazine — For Web Designers And Developers

08 Dec

How To Overcome Data Onboarding Challenges For Software Products

by TBSCategories: News

Companies willing to pay good money for a new piece of software are most likely not starting from scratch. They’re running an established business, with well-built and documented processes. So, they have tons of data to carry over.

As a result, the decision to bring a new app into the fold is not one they take lightly. Internal processes need to change. Getting the team to adopt the new solution can take time. Integrating it with existing systems and external tools can be a problem. Oh yeah, and there’s the matter of compliance to worry about, too.

This means there’s a lot of pressure on new software to provide a top-notch experience from the get-go. Fail to provide companies with a simple and intuitive way to onboard their data and you can expect high rates of customer churn as a result.

If you’re designing a product that needs data from customers in order to be of any value, here’s what you need to know about building out your data onboarding process.

How Data Onboarding Correlates With User Satisfaction

Business software is essentially just an empty box waiting to be filled with its users’ data. Without the ability to flawlessly onboard users’ data, the software essentially becomes useless.

Let’s look at what happens when you get the data onboarding process right.

End User Benefits

If you can nail the data onboarding piece, expect your end users to reap the following benefits:

  • They’ll be more confident in their decision.
    With complete and accurate data transferred into your software, users actually see how valuable it is soon after signing up. This leaves little room for second-guessing their decision, which leads to greater satisfaction overall with the product, and ultimately more money for your business.
  • You’ll get greater team buy-in.
    A positive data onboarding experience lets customers use your product faster, reducing the time needed for them to get value. So, really, data onboarding sets the stage for how your customers and their team will view the rest of your app.
  • They’ll experience more success with the software.
    Since users won’t have to stress about data formatting and cleanup or troubleshooting error-ridden import processes, they can get more out of the product and its features.

Software Developer Benefits

The software provider (you and/or your client) benefits, too:

  • Improve user satisfaction.
    Your end users don’t need to be technical wizards to figure out how to onboard data into your product. When you make light work of this, you reduce churn, attract more users and retain more loyal users over the long-term.
  • Spend less time on customer service.
    You can stop worrying about having to support a faulty data onboarding process as well as taking over tasks like data formatting and validation for your users. Instead, put your time and energy towards building better relationships with customers instead of putting out fires all the time.
    Kelly Abbott, Co-Founder and CTO of Tablecloth, can attest to this:

    “We have cut the amount of time we spend wrangling with files by 95%. We basically had all hands working to solve those problems at times.”

  • Have greater confidence in your product.
    When you have a data onboarding solution that’s flexible and powerful, you don’t have to restrict what data your users can or can’t import. It’s no longer a limitation.
    As Abbott explains:

    “It has made us more contemplative about the data we are asking clients for. We no longer have to avoid asking for data that may require too much time to fix. Flatfile eliminates that problem and has improved our willingness to experiment with different types of data we can incorporate into our analyses. The more time we spend tinkering with different data types, the more likely we are going to uncover the insight that produces additional value in the marketplace. That is indispensable for a startup like us.”

  • Save money.
    Although you’ll have to spend money on a third-party data onboarding solution, you’ll save your company the time and money otherwise spent trying to manage a custom-built data importer, onboarding process and client relationships. (Tablecloth, for instance, saved tens of thousands of dollars when they adopted Flatfile.)

The Challenges Of Data Onboarding For Software Products

Let’s have a look at the common challenges in data onboarding and how Flatfile Concierge removes them:

Challenge #1: There’s A Lot Of Data To Aggregate

When signing up for new business software, users probably expect to do a little work upfront, like filling in basic account information, configuring settings and adding users. The last thing you want to do is surprise them with a data importer that’s going to cause more work for them.

Let’s say, for example, you’ve built a CRM.

Unless the software targets startups and other new businesses, users are going to have a ton of external data to bring along. For instance:

  • Contact info for clients, prospects, vendors, partners and team members;
  • Existing customer data like account and sales history;
  • Prospect data like communication history;
  • Sales pipeline details;
  • Team and individual goals and metrics.

Unless your CRM directly integrates with every one of your users’ previous CRMs, how are they going to move this data over? Copy and paste? CSV templates?

Concierge allows companies to import ‘data model’ templates based on spreadsheet columns for easy setup. Customers would then import data that automatically matches this model with no code needed. (Image source: Flatfile)

Plus, you have to think about all of the other sources a CRM pulls info in from. Payment gateways. Spreadsheets that live on a sales team’s drive. Signed contracts that have been emailed or faxed to your company. There’s a lot of data coming from different places and people.

The Fix

There are a number of things Flatfile Concierge does to fix this problem.

For starters, it allows data to be imported from a variety of file types:

  • CSV,
  • TSV,
  • XLS,
  • XML,
  • And more.

With this kind of flexibility, your users won’t have to worry about transferring data to one specific file type and then cleaning up errors that occur during the transfer. Flatfile Concierge can handle various file types, of varying data types, and easily validate it all.

Another thing to think about is how your software is going to track and organize each imported file and its corresponding data.

What Flatfile allows your users to do is create collaborative workspaces to place data in. When a team member adds new data to the workspace, a record is captured containing the:

  • Date of upload,
  • File name,
  • User who submitted the data,
  • Number of rows added,
  • Version history,
  • Upload errors.

Flatfile Concierge animation demonstrating notifications for when spreadsheets are imported. (Image source: Flatfile)

This will keep things organized while also keeping everyone accountable to the data they contribute. And with this information readily available from a centralized dashboard, there’ll be no secret as to what’s been uploaded, by whom and when. Import errors can also be fixed collaboratively, without the need to re-upload spreadsheet data.

Challenge #2: Data Is Imported In A Variety Of States

When you give your software users the ability to transfer their data into your product, there’s not a lot you or the software team can do in terms of formatting or cleaning up end users’ data beforehand. Nor should you have to. Your job is to ensure customers see the value in the software; not to struggle with importing data.

You could give them a spreadsheet template, but that would require them to spend time reformatting all their data. You could point them to the knowledgebase, but, again, that assumes that your end users will be willing to do that extra work.

In reality, your users are going to be in a hurry to get inside the new software and get to work. They’re not going to stop to deal with this. That’s the software’s job.

However, many data onboarding solutions don’t handle messy spreadsheets very well. Not only do they have a hard time recognizing what some of the data is (often because the data model doesn’t match their own), but then the application refuses to accept certain spreadsheet columns.

Even if it’s the end user’s fault for not properly organizing or labeling their data or teaching their team how to do so (or just not knowing what to do in the first place), who do you think they’re going to blame in the end when their data won’t import?

The Fix

Flatfile Concierge’s importer is AI-powered, which means that your software (and data importer) really can do the work for your end users.

Using advanced validation logic, the data importer can figure out what the data is and where it goes.

While Flatfile will automatically match columns and corresponding data to your software’s actual data fields, users get a chance to confirm that’s the case before allowing it into the system:

Before this happens, you can do a little work on the backend to ensure that Flatfile knows what to do with your users’ data:

  • Create target data models so Flatfile can navigate complex spreadsheet formats and datatypes your users will likely try to import.
  • Create a template with validation rules so Flatfile’s AI knows exactly how to map everything out.
  • Validate imported data against other databases to help the importer contextualize, validate and clean up the data over time.

Once you’ve done that upfront work, the rest is easy.

The bulk of the work will be done by Flatfile Concierge when it transforms imported data into something clean and useful. In fact, about 95% of imported columns will automatically map to your software thanks to Flatfile’s machine learning and fuzzy matching system.

The end user will have the opportunity to review the parts of their data that contain errors. If they find any, they can repair the errors inside Flatfile, rather than have to fix it in a spreadsheet and re-import.

Challenge #3: Getting And Tracking Data From Multiple Users

When there are a lot of cooks in the kitchen, there are a number of things that can go wrong.

Data can sometimes live on team members’ computers, or worse, sent over email, which can be a huge security concern for sensitive data. This can happen if users aren’t given access to the software platform or find the data importer too intimidating to use.

On the flip side of that, with the wrong data onboarding process, it could become like a free-for-all where people add whatever the heck they like to the company’s data. While the data does get imported, there’s no review framework so the company’s database is filled with errors and duplicate entries.

Your end users need to be able to maintain order, control and security when dealing with something as serious as company data — especially if you want your software to be usable.

The Fix

Flatfile Concierge has designed the data onboarding process to be a collaborative one.

As you can see, company admins can invite specific collaborators (i.e. customers) to add data to their workspaces. But this isn’t a blanket invitation to import data.

Admins have the ability to create an approval process. They get to:

  • Ask for specific data sets from team members.
  • Control which workspaces they’re allowed to import data to.
  • Review all data submissions before flowing the approved data into the platform.

Admins can also import data on the customer’s behalf. Flatfile Concierge ensures that data onboarding is never a dead-end for customers.

Not only does this ensure that the right data ends up in the software, but the controlled flow means the data will end up being cleaner and more accurate, too. All of this, while providing a seamless data onboarding experience for users.

Challenge #4: Data Security Is Always A Concern

When it comes to web and app development, user privacy and security are top priority. If our customers and visitors don’t trust that their information is safe from prying eyes (and isn’t being sold off to advertisers), they’re going to stop using our solutions in the first place.

The same thing happens with software — though it’s not just the company’s personal data they have to worry about securing.

Often, when companies import data into software (like the CRM example), they’re importing their customers’ private and sensitive data. Allow that to be compromised and you can kiss your software goodbye.

So, yes, the software itself needs to be secured. That’s a given. But so, too, does your data onboarding process. It’s a huge point of vulnerability if left unchecked.

The Fix

The first thing Flatfile Concierge does is to encourage users to move away from sharing sensitive data over email, FTP, and other unsecured platforms by providing a user-friendly data onboarding solution.

The second thing it does is provide an authenticated and compliant workspace for users to import, validate, and post their data to your software.

Here’s how Flatfile Concierge secures its workspaces:

  • Each collaborator enters the data importer through an authenticated invitation.
  • Data is encrypted in transit and stored in an encrypted Amazon S3 bucket.
  • The data onboarding platform is 100% GDPR compliant.
  • Flatfile is HIPAA and SOC2 compliant and can adjust for other compliance requirements as needed.

In addition, once data is successfully migrated into your application, it’s deleted from Flatfile. This way, you only have to worry about securing your data within your software and not on previous platforms it’s touched.

Wrapping Up

With an insufficient or error-prone data onboarding process, you, the software provider and its end users are going to spend too much time manually cleaning and validating spreadsheets. This won’t just happen during the initial user signup either. If the data importer isn’t up to the task, you’re all going to be throwing away a ton of time and resources every time data needs to be uploaded or transferred into the platform from existing customers.

Of course, this all assumes that your importer can even get user data into the software. (Sadly, this happens with too many custom-built solutions.)

Needless to say: Your data onboarding process must be flawless for your team and customers. It’s the only way to keep user churn rates low and user satisfaction high.

Data onboarding is a really complex process to handle. Save yourself the trouble in trying to develop your own data onboarding solution and the time trying to troubleshoot the problems with it. With an AI-powered data importer like Flatfile Concierge, everything’s taken care of for you.

Articles on Smashing Magazine — For Web Designers And Developers

logo-red-11
14 Jul

How To Create A Custom React Hook To Fetch And Cache Data

by TBSCategories: News

How To Create A Custom React Hook To Fetch And Cache Data

How To Create A Custom React Hook To Fetch And Cache Data

Ademola Adegbuyi

If you are a newbie to React Hooks, you can start by checking the official documentation to get a grasp of it. After that, I’d recommend reading Shedrack Akintayo’s “Getting Started With React Hooks API”. To ensure you’re following along, there is also an article written by Adeneye David Abiodun that covers best practices with React Hooks which I’m sure will prove to be useful to you.

Throughout this article, we’ll be making use of Hacker News Search API to build a custom hook which we can use to fetch data. While this tutorial will cover the Hacker News Search API, we’ll have the hook work in a way that it will return response from any valid API link we pass to it.

Best Practices With React

React is a fantastic JavaScript library for building rich user interfaces. It provides a great component abstraction for organizing your interfaces into well-functioning code, and there’s just about anything you can use it for. Read more articles on React →

Fetching Data In A React Component

Before React hooks, it was conventional to fetch initial data in the componentDidMount() lifecycle method, and data based on prop or state changes in componentDidUpdate() lifecycle method.

Here’s how it works:

componentDidMount() {   const fetchData = async () => {     const response = await fetch(       `https://hn.algolia.com/api/v1/search?query=JavaScript`     );     const data = await response.json();     this.setState({ data });   };      fetchData(); }   componentDidUpdate(previousProps, previousState) {     if (previousState.query !== this.state.query) {       const fetchData = async () => {         const response = await fetch(           `https://hn.algolia.com/api/v1/search?query=$ {this.state.query}`         );         const data = await response.json();         this.setState({ data });       };        fetchData();     }   }

The componentDidMount lifecycle method gets invoked as soon as the component gets mounted, and when that is done, what we did was to make a request to search for “JavaScript” via the Hacker News API and update the state based on the response.

The componentDidUpdate lifecycle method, on the other hand, gets invoked when there’s a change in the component. We compared the previous query in the state with the current query to prevent the method from getting invoked every time we set “data” in state. One thing we get from using hooks is to combine both lifecycle methods in a cleaner way — meaning that we won’t need to have two lifecycle methods for when the component mounts and when it updates.

Fetching Data With useEffect Hook

The useEffect hook gets invoked as soon as the component is mounted. If we need the hook to rerun based on some prop or state changes, we’ll need to pass them to the dependency array (which is the second argument of the useEffect hook).

Let’s explore how to fetch data with hooks:

import { useState, useEffect } from 'react';  const [status, setStatus] = useState('idle'); const [query, setQuery] = useState(''); const [data, setData] = useState([]);  useEffect(() => {     if (!query) return;      const fetchData = async () => {         setStatus('fetching');         const response = await fetch(             `https://hn.algolia.com/api/v1/search?query=$ {query}`         );         const data = await response.json();         setData(data.hits);         setStatus('fetched');     };      fetchData(); }, [query]);

In the example above, we passed query as a dependency to our useEffect hook. By doing that, we’re telling useEffect to track query changes. If the previous query value isn’t the same as the current value, the useEffect get invoked again.

With that said, we’re also setting several status on the component as needed, as this will better convey some message to the screen based on some finite states status. In the idle state, we could let users know that they could make use of the search box to get started. In the fetching state, we could show a spinner. And, in the fetched state, we’ll render the data.

It’s important to set the data before you attempt to set status to fetched so that you can prevent a flicker which occurs as a result of the data being empty while you’re setting the fetched status.

Creating A Custom Hook

“A custom hook is a JavaScript function whose name starts with ‘use’ and that may call other Hooks.”

— React Docs

That’s really what it is, and along with a JavaScript function, it allows you to reuse some piece of code in several parts of your app.

The definition from the React Docs has given it away but let’s see how it works in practice with a counter custom hook:

const useCounter = (initialState = 0) => {       const [count, setCount] = useState(initialState);       const add = () => setCount(count + 1);       const subtract = () => setCount(count - 1);       return { count, add, subtract }; };

Here, we have a regular function where we take in an optional argument, set the value to our state, as well as add the add and the subtract methods that could be used to update it.

Everywhere in our app where we need a counter, we can call useCounter like a regular function and pass an initialState so we know where to start counting from. When we don’t have an initial state, we default to 0.

Here’s how it works in practice:

import { useCounter } from './customHookPath';  const { count, add, subtract } = useCounter(100);  eventHandler(() => {   add(); // or subtract(); });

What we did here was to import our custom hook from the file we declared it in, so we could make use of it in our app. We set its initial state to 100, so whenever we call add(), it increases count by 1, and whenever we call subtract(), it decreases count by 1.

Creating useFetch Hook

Now that we’ve learned how to create a simple custom hook, let’s extract our logic to fetch data into a custom hook.

const useFetch = (query) => {     const [status, setStatus] = useState('idle');     const [data, setData] = useState([]);      useEffect(() => {         if (!query) return;          const fetchData = async () => {             setStatus('fetching');             const response = await fetch(                 `https://hn.algolia.com/api/v1/search?query=$ {query}`             );             const data = await response.json();             setData(data.hits);             setStatus('fetched');         };          fetchData();     }, [query]);      return { status, data }; };

It’s pretty much the same thing we did above with the exception of it being a function that takes in query and returns status and data. And, that’s a useFetch hook that we could use in several components in our React application.

This works, but the problem with this implementation now is, it’s specific to Hacker News so we might just call it useHackerNews. What we intend to do is, to create a useFetch hook that can be used to call any URL. Let’s revamp it to take in a URL instead!

const useFetch = (url) => {     const [status, setStatus] = useState('idle');     const [data, setData] = useState([]);      useEffect(() => {         if (!url) return;         const fetchData = async () => {             setStatus('fetching');             const response = await fetch(url);             const data = await response.json();             setData(data);             setStatus('fetched');         };          fetchData();     }, [url]);      return { status, data }; };

Now, our useFetch hook is generic and we can use it as we want in our various components.

Here’s one way of consuming it:

const [query, setQuery] = useState('');  const url = query && `https://hn.algolia.com/api/v1/search?query=$ {query}`; const { status, data } = useFetch(url);

In this case, if the value of query is truthy, we go ahead to set the URL and if it’s not, we’re fine with passing undefined as it’d get handled in our hook. The effect will attempt to run once, regardless.

Memoizing Fetched Data

Memoization is a technique we would use to make sure that we don’t hit the hackernews endpoint if we have made some kind of request to fetch it at some initial phase. Storing the result of expensive fetch calls will save the users some load time, therefore, increasing overall performance.

Note: For more context, you could check out Wikipedia’s explanation on Memoization.

Let’s explore how we could do that!

const cache = {};  const useFetch = (url) => {     const [status, setStatus] = useState('idle');     const [data, setData] = useState([]);      useEffect(() => {         if (!url) return;          const fetchData = async () => {             setStatus('fetching');             if (cache[url]) {                 const data = cache[url];                 setData(data);                 setStatus('fetched');             } else {                 const response = await fetch(url);                 const data = await response.json();                 cache[url] = data; // set response in cache;                 setData(data);                 setStatus('fetched');             }         };          fetchData();     }, [url]);      return { status, data }; };

Here, we’re mapping URLs to their data. So, if we make a request to fetch some existing data, we set the data from our local cache, else, we go ahead to make the request and set the result in the cache. This ensures we do not make an API call when we have the data available to us locally. We’ll also notice that we’re killing off the effect if the URL is falsy, so it makes sure we don’t proceed to fetch data that doesn’t exist. We can’t do it before the useEffect hook as that will go against one of the rules of hooks, which is to always call hooks at the top level.

Declaring cache in a different scope works but it makes our hook go against the principle of a pure function. Besides, we also want to make sure that React helps in cleaning up our mess when we no longer want to make use of the component. We’ll explore useRef to help us in achieving that.

Memoizing Data With useRef

“useRef is like a box that can hold a mutable value in its .current property.”

— React Docs

With useRef, we can set and retrieve mutable values at ease and its value persists throughout the component’s lifecycle.

Let’s replace our cache implementation with some useRef magic!

const useFetch = (url) => {     const cache = useRef({});     const [status, setStatus] = useState('idle');     const [data, setData] = useState([]);      useEffect(() => {         if (!url) return;         const fetchData = async () => {             setStatus('fetching');             if (cache.current[url]) {                 const data = cache.current[url];                 setData(data);                 setStatus('fetched');             } else {                 const response = await fetch(url);                 const data = await response.json();                 cache.current[url] = data; // set response in cache;                 setData(data);                 setStatus('fetched');             }         };          fetchData();     }, [url]);      return { status, data }; };

Here, our cache is now in our useFetch hook with an empty object as an initial value.

Wrapping Up

Well, I did state that setting the data before setting the fetched status was a good idea, but there are two potential problems we could have with that, too:

  1. Our unit test could fail as a result of the data array not being empty while we’re in the fetching state. React could actually batch state changes but it can’t do that if it’s triggered asynchronously;
  2. Our app re-renders more than it should.

Let’s do a final clean-up to our useFetch hook.,We’re going to start by switching our useStates to a useReducer. Let’s see how that works!

const initialState = {     status: 'idle',     error: null,     data: [], };  const [state, dispatch] = useReducer((state, action) => {     switch (action.type) {         case 'FETCHING':             return { ...initialState, status: 'fetching' };         case 'FETCHED':             return { ...initialState, status: 'fetched', data: action.payload };         case 'FETCH_ERROR':             return { ...initialState, status: 'error', error: action.payload };         default:             return state;     } }, initialState);

Here, we added an initial state which is the initial value we passed to each of our individual useStates. In our useReducer, we check what type of action we want to perform, and set the appropriate values to state based on that.

This resolves the two problems we discussed earlier, as we now get to set the status and data at the same time in order to help prevent impossible states and unnecessary re-renders.

There’s just one more thing left: cleaning up our side effect. Fetch implements the Promise API, in the sense that it could be resolved or rejected. If our hook tries to make an update while the component has unmounted because of some Promise just got resolved, React would return Can't perform a React state update on an unmounted component.

Let’s see how we can fix that with useEffect clean-up!

useEffect(() => {     let cancelRequest = false;     if (!url) return;      const fetchData = async () => {         dispatch({ type: 'FETCHING' });         if (cache.current[url]) {             const data = cache.current[url];             dispatch({ type: 'FETCHED', payload: data });         } else {             try {                 const response = await fetch(url);                 const data = await response.json();                 cache.current[url] = data;                 if (cancelRequest) return;                 dispatch({ type: 'FETCHED', payload: data });             } catch (error) {                 if (cancelRequest) return;                 dispatch({ type: 'FETCH_ERROR', payload: error.message });             }         }     };      fetchData();      return function cleanup() {         cancelRequest = true;     }; }, [url]);

Here, we set cancelRequest to true after having defined it inside the effect. So, before we attempt to make state changes, we first confirm if the component has been unmounted. If it has been unmounted, we skip updating the state and if it hasn’t been unmounted, we update the state. This will resolve the React state update error, and also prevent race conditions in our components.

Conclusion

We’ve explored several hooks concepts to help fetch and cache data in our components. We also went through cleaning up our useEffect hook which helps prevent a good number of problems in our app.

If you have any questions, please feel free to drop them in the comments section below!

References

Smashing Editorial (ks, ra, yk, il)


Articles on Smashing Magazine — For Web Designers And Developers

Newer postsOlder posts
×

Powered by WhatsApp Chat

WhatsApp Chat is free, download and try it now here!

× Chat with Us