Custom imports are a powerful and flexible tool to ingest many kinds of data from custom sources.
As a modern newsroom, you likely have multiple content streams and each of those streams are sending you valuable data. Custom imports allow you to gather as much of that data as you need into one place.
Often, this feature is used to import real-time or historical data that originates from data services other than common analytics services or RSS feeds. The feature will programmatically retrieve an import file from a source of your choice and add its contents to MFN according to the settings you configure.
There are three types of custom imports:
- Analytics: Import metrics for each article. Columns in the spreadsheet can be matched to any metric that exists in the application. Data in a custom import should always represent the all-time total engagement with the given article. Data imported from a custom import will completely replace and override any pre-existing data for an article with the number specified in the import file.
- Content: As an alternative to RSS feeds, a custom import can provide similar information and it will be imported through the exact same methods that feeds are imported through, including autotagging of new content.
- Authors: A spreadsheet of author names to import as known author identities. Columns can also specify the Author Type and/or the Team that the author should be assigned to. This is usually the most efficient way to initially set up all of a property’s authors, author types and teams in one step.
See below for applicable importing file formats, data sources, and instructions to get started.
Supported File Formats and Data Sources
File Formats:
- CSV: Delimited by a comma or tab
Data Sources: (each option requires you to provide permission, access keys or logins)
- URL (e.g., Link to Google Sheet with "Anyone with the link" permission)
- FTP or SFTP
- Amazon S3 bucket
- Google Cloud Storage bucket (Authentication instructions)
How to access Custom Imports
Note: Must be logged into an account that has “admin” access in order to view and manage custom imports.
Steps to reach the Custom Imports page:
- Go to your account’s Settings page
- In the Other Settings box, click “Custom imports”
- On this page, you can Add, Edit, and Delete custom imports
Preparing for a Custom Import
General Requirements
Apply to all custom import types
- Data Source: Must be one of the supported data sources (See above)
- Schedule: Frequency and time of day should this feed be retrieved; Options include regular or one-time, per day or per week.
- Timeframe: A specific time frame for MFN to look back when retrieving data from the file.
- File Format: Must be one of the supported formats (See above)
- File Contents: The data included in the file must be organized and named consistently. See below for specific requirements based on the type of import file.
Metrics for News and Source Matters considers all data within the file you provide. If necessary, filter the data prior to adding it in the file, such as excluding content or data points that you do not want to use.
Import File Settings
As you change the Type field for the import, “field settings” will appear for that type at the bottom of the Custom Import form. The field settings refer to the columns in the import file, which each represent unique data to import such as article titles for a Content import and pageviews for an Analytics import. The sections below are specific requirements for each type’s field settings.
The name referenced for each data column is a suggestion and may be named differently in your file. When creating a custom import, you will need to specify the exact name used in your file. If the name isn't typed into the field settings, the column in the file will not be imported.
Field settings for Content import files
Copy sample file: Content-only import, Content & Analytics import
Acceptable data columns:
- Article URL: Required. Full web URL for an article.
- Publication Date: Required. The date that the article was published. If a time is provided, it must be separated by either a whitespace, period, or comma. Note: MFN handles date ranges by publication date of the article, rather than by traffic date. The file to import to show all the applicable data for a given article, and to include stories published during the reporting period.
- GUID field: Not required but highly recommended. The tag used for the unique ID or Canonical URL of the article. Improves reliability for matching articles with their data.
- GUID prefix: May be required if using the GUID field. If the GUID provided in the GUID field includes a prefix to the ID, the prefix needs to be ignored. Add the prefix here so the system knows what to ignore.
- Author: Not Required. The full name of the author who wrote the article. Note: If not provided, "Byline Wrappers" must be provided in the Page Scraping page in Setup.
- Headline: Required. The full title of the article.
- Full Text: Not Required. The entire content of the article. If not provided in data sheet or RSS feed, "Content Wrappers" must be provided in the Page Scraping page in Setup.
- Summary: Not Required. An excerpt of the article's text. If not provided, MFN will use the first 100 characters of the Full Text for the article.
Field settings for Analytics import files
Applies to Metrics for News accounts only
Copy sample file: Analytics-only import, Content & Analytics import
Acceptable data columns:
- Article URL: Required. Full web URL for an article.
- Word Count: Not Required. The number of words in the article body.
- Average Time on page: Not Required. The average time people spend on the article’s page.
- Pageviews: Not Required. The number of total pageviews that the article has received.
Additional data columns may be included depending on the Custom Metrics or Audiences that you have set up on your account. For a full list of applicable columns, visit the Custom Import form (Click “Add” or “Edit”), select the Type dropdown to be “Analytics only” or “Both analytics and content” then scroll down to the “field settings” section .
Field settings for Author import files
Copy sample file: Author import
Acceptable data columns:
- First Name: The first name of the author as it appears on your site.
- Last Name: The last name of the author as it appears on your site. This, combined with first_name, will be used to identify authors.
- Full Byline: The full name of the author as it appears on your site. This will be used to identify authors if First Name and Last Name don't find any results.
- Author Type: The author type will be associated with their level of access (e.g., staff, freelance, editor).
- Team: The team that the author is on. (e.g., Business, Sports). If you would like to add multiple teams to one author, list those teams in the same cell separated by a pipe ("|").
Adding a custom import
Go to the "Custom imports" page in Metrics for News and click "Add New Import."
Set Up a Custom Import
For the first section, "Set up custom import," please enter:
-
- Name: Name that describes the import.
- Publication: Select the publication associated with this import.
- Import frequency: Select one - this can be changed any time:
- Manual or one time (contact us to import)
- Daily (enter the time of day)
- Weekly
- Type: Select the option that best fits.
- Authors (when batch importing Author profiles)
- Content only
- Analytics only
- Both analytics and content
- Oldest content to update: Select the option that best finishes the sentence.
- The import will only modify analytics data for content that was published within the last... (select an option from the dropdown menu).
- And the import will only modify analytics data for content that was published on or after [DATE] (use the calendar picker to select a date; or leave blank for no restriction).
- File URL: If you are using an import method that provides a file URL such as a CSV, paste the URL here.
- A button will appear asking if you want to convert this to a CSV. Click the button to agree.
-
- Data format: Leave at default unless you are using a different format.
- Field delimiter: Leave at default unless you are using a different delimiter.
- Data compression format: Leave at default unless you are using a different format.
Import Methods
Fill in the fields below for the import method you are using. Follow the field setting details from above. If you are not using a particular import method, skip that section.
- SFTP / FTP file-download settings
- AWS S3 file-download settings
- Google Cloud Storage file-download settings
Content field settings
In each field, enter the name of the matching field from your data source. For example, if you are using our sample spreadsheet, you would enter "article_url" for the Article URL field.
Analytics field settings
In each field, enter the name of the matching field from your data source. For example, if you are using our sample spreadsheet, you would enter "pv_total" for the total pageviews field.
Want to add a metric but don't see a field for it? Add a custom metric!
Click save at the bottom to save your work. Once your import is ready, it will run at the frequency you selected.
If the import frequency of your custom import is set to manual, then please submit a ticket to let us know to run the import. We will let you know when the import has completed, though you may see the data appear in MFN before we are able to update you.
Keep in Mind
Here are a few tips to keep in mind about custom imports.
- Remember to save your work as you go along. You can always edit a saved custom import anytime.
- The system will import from the first tab only of a CSV file. You can add additional tabs to the spreadsheet for your own use, but for any content that needs to be imported, ensure the data is on the first tab.
- Looking to add data from your syndicated partners? Check out our article on configuring syndicated views.
- If you no longer need a custom import, you can delete it or you can set it to manual so that it is inactive. If you are setting an import to manual for archiving purposes, we recommend adding "Archived" to the name.
- If you have more than one custom import running automatically, be careful that they are not overwriting each other. Submit a support ticket if you'd like us to check.