Data Sandbox

Expand all | Collapse all

Does your data product pass its FIT test?

  • 1.  Does your data product pass its FIT test?

    Pitney Bowes
    Posted 06-03-2019 10:24

    A first impression is a lasting impression…

    Whenever I start using a new data product, I put it through my own version of a data fitness test – I call it the FIT test (i.e. the First Impressions Test).  My 3-part First Impressions Test can quickly tell me whether the Data Product likely has decent quality and is worth examining in depth.

    While not part of my FIT test, step zero is retrieving the product and its documentation. A poor experience, such as having to access files from a problematic FTP location or documentation that is emailed separately from the product files, can be indicative of problems to come. The Pitney Bowes Software & Data Marketplace is a positive experience for me – the data products and their documentation are packaged nicely together.  A challenging experience retrieving the files and unpacking them can be a red flag.  

    At this point I conduct my 3-part First Impressions Test:

    1. I read the Product Documentation. Yes, I know that's quaint and very few people read the documentation, but it gives you an immediate sense of what to expect for the rest of the product experience. I look at field definitions, logical layout of the product and any obvious problems in the product design.  Examples of poor product documentation include the lack of explanation of data fields, use acronyms without explaining them, or providing field descriptions that are anything but descriptive... A review of the documentation is especially helpful if you know nothing about the product and are experiencing it as a first-time user. Helpful documentation includes products statistics, FAQs, and known quality issues. A product manager should read their own documentation with this perspective.
    2. Second, I open the Data Product and sanity check the data consistency with the documentation. Do the field domains in the data product agree with the documentation? Are there surprise files or fields in the data product that were not mentioned in the documentation?  A quick set of pivot tables in Excel can find outliers in the data (for example, nulls in fields that otherwise should not have null values per the product design).  Sorts and a few SQL queries can tell you whether the data has completeness of its fields and if there are gross data correctness issues.
    3. Third, I look at the data. My passion is for spatial data products, so next I look at the data in an application like MapInfo Pro.  Looking at the data with a critical eye is very important and often overlooked (pun intended).  Zoom to the extent of the data and look at its spatial distribution for a moment – does it look reasonable?  Are there large holes in the coverage where you expect data?  Next, do a sanity check on the data contents by looking at a few places you are intimately familiar with.  The best places to look at are places you likely know much better than the data provider (e.g. where you grew up, where you lived or worked for years, where you have a 2nd home, where you live now) and compare the data contents with your own knowledge of the area.  Does the data agree with your own knowledge?  Does the data look current, complete and correct?  If not, it probably has problems in other locations where you lack that personal knowledge…

    Doing this 3-part First Impressions Test gives you a sense of the product and whether it's likely going to meet your needs. Its not exhaustive but its a quick and effective sanity check and can help you determine whether it is worth your time and effort to do a more thorough examination of the product.



    ------------------------------
    Tom Gilligan
    Pitney Bowes Software, Inc.
    White River Junction, VT, USA
    ------------------------------


  • 2.  RE: Does your data product pass its FIT test?

    Pitney Bowes
    Posted 06-13-2019 07:31
    Tom,
    I enjoyed reading this and think you are right on - before you use or buy any type of data, you need to perform some type of sanity check on the data before you use/buy it. I think many people will find this post and the steps you outline helpful!
    Colleen

    ------------------------------
    Colleen Reed
    Pitney Bowes
    White River Junction VT
    ------------------------------



  • 3.  RE: Does your data product pass its FIT test?

    Pitney Bowes
    Posted 06-13-2019 14:47
    ​Tom,

    I agree, great post and simple yet detailed advice.  I can't even imagine how many times over my career I have answered questions about data, that the user could have simply answered themselves if they would have read the documentation.  I think this is due to a common problem of good data often being accompanied with poorly constructed, or overly complicated documentation.  For data users this has created an environment where it seems easier to ask a support group or form than search overly complex product documentation to find the answers they need.  I appreciate simple and well organized product documentation that makes it easy to use and implement into your intended solution.  A great data product is only as good as the supporting materials it includes.

    Dylan

    ------------------------------
    Dylan Conrad
    Product Manager
    ------------------------------



  • 4.  RE: Does your data product pass its FIT test?

    Pitney Bowes
    Posted 06-13-2019 15:22
    Three great steps Tom. My general approach when I get my hands on some new data is probably a bit more like a kid in a candy store:

    kid in candy store
    My steps are closer to:

    1. open the data in the relevant software package
    2. load some other data in with it for background context - such as an imagery, administrative or roads layers
    3. select some data and see what it contains, do some themes and look around
    4. get frustrated that I am not linking things together correctly or realize that I need to know more about the data model - pushing me to the documentation
    Once I look at the documentation I generally need to repeat number 3, or find the documentation lacking and get frustrated.
    5. turn to analytics tools to understand completeness of fields

    Your approach makes so much more sense than my general desired to play with the data - I will try to control my desire to head straight to playing next time.

    ------------------------------
    Gerry Stanley
    Pitney Bowes Australia Pty Ltd
    Macquarie Park
    ------------------------------



  • 5.  RE: Does your data product pass its FIT test?

    Pitney Bowes
    Posted 06-18-2019 02:19
    Its really a good read. I will make sure I always perform FIT test while testing our data products.

    ------------------------------
    Abhimanyu Jha
    Knowledge Community Shared Account
    Shelton CT
    ------------------------------