Search the Omeda Knowledge Base
Data Loader FAQs
GENERAL
What format does my file need to be in order to upload to Data Loader?
Your files need to be in .CSV format.
Can I see everyone’s files, or just my own?
All uploaded files will appear in the ‘Files’ section regardless of who uploaded them. If you are looking to see files uploaded by a specific person, you can search for their username using the ‘Filter’ option at the top of the screen.
Note: The only exception to this is if you load files in a certain profile – such files are only available in that profile and won’t appear in other lists.
How many records can I process in one file?
There is no limit on the number of records/rows a file can have. However, with files over 1k rows, we recommend uploading a ‘demo’ file with the first 10-50 rows and processing it to ensure everything maps and matches as intended before you process the larger file.
What type of data/products can I map to and process in Data Loader?
Currently you can map to any of the following data sets:
- General Contact & Address Info
- Demographics
- Behaviors
- Newsletter Products
- Deployment Types
How long does it take for a file to process?
The processing time will vary depending on a few factors:
- The # of rows in your file
- The # of headers in your file
- The complexity of mapping
- The validation rules you are applying
Processing times can be as short as a few minutes for smaller files and take up to an hour or more for larger files. You can track all file progress using the ‘Queues’ section.
Can I customize the ‘Data Source’ dropdown for my database?
This drop down is reflective of the standard Magazine Source categories and is not customizable.
Is there a way to see what fields I haven’t mapped yet?
Yes, when you are in the ‘General Mapping’ section, there will be a box in the lower left that tallies how many un-mapped fields your file has.
My file is missing a column – do I have to manually add it before I upload the file?
No, if you would like to add one or more columns that have the same data for all the incoming rows (e.g. same promo code, product mapping, same behavior etc) you can add a ‘Constant’ in the General Mapping step.
Check out ‘Adding File Constants‘ for more details.
Do I have to include both a Country and Country Code if I want to map a Country field?
No, Data Loader will populate the Country Code when files are processed containing the correct Country Name and the same will be true in reverse. If a file contains the accurate Country Code, Country Name will be populated.
Do I have to include both a Region and Region Code if I want to map a Region?
No, Data Loader will populate the Region Code when files are processed containing the correct Region Name and the same will be true in reverse. If a file contains the accurate Region Code, Region Name will be populated.
Can map Behaviors Attributes without also mapping a Behavior?
No, Behavior Attributes require a Behavior to be mapped to be committed to the database. To better guide the user though this an error level validation will be triggered when a user maps any of the Behavior Attribute fields but neglects to map a Behavior field. This error will prevent the user from processing the file with just Behavior Attributes mapped.
MAPPING
Do I have to map all incoming fields?
No, one of the benefits of using Data Loader is that you can leave off mapping certain columns without manually removing them from your source file. If it’s not mapped, we simply pass over it during processing and continue with the rest of the data.
Do I have to map an incoming column if it’s blank?
No, if the entire column is blank, you can leave it un-mapped in the General Mapping section and it will be passed over once you process the file.
Are certain fields required?
No, currently there are no required fields that you need to include in your files or mappings.
What products can I map to?
The Data Loader supports mappings to all of your active Newsletter Products and associated Newsletter classes.
How do I map a promo code?
On the ‘General Mappings’ page, you have the option to select ‘Promo Code’ from the available mappings.
If I don’t have a promo code, does Data Loader automatically apply one?
No, if your file doesn’t have a promo code and it’s not added as a constant, no promo code will be added to your records.
How do I attach a verification date to the records?
Selecting ‘Transaction Date’ in the General Mapping step will write to the verification date for the newsletter product in the database.
Note: The verification date will only use the transaction date field when you have a product mapping present.
What happens if I don’t have any dates mapped?
If you are mapping to a product and don’t have a datefield present, the Data Loader will apply the date the file was processed to the incoming records.
How do I map to a profile?
Records will automatically pull into a profile based on their product, demographic, and behavioral mappings. You cannot map to a profile directly.
How do I map to events?
This will vary depending on how your database is set up. Most often, you can map to an event using the behavior fields (e.g. attended or registered X event)
If you have any questions about mapping your event data, please contact your audience services team for direction.
Can I just add new responses to existing customers in my database, or will it overwrite existing answers/preferences?
For demographics, behaviors, products, and deployment types you can choose to Add Incoming Only OR Update or Add Incoming:
- Add Incoming Only will add the data to the records only if they didn’t have a response.
- Update or Add Incoming will overwrite existing response with the new data AND it will add data to records that didn’t have a response previously.
In a multi-value field, can I ignore fields?
Yes, you don’t have to map all of the incoming values on your source file. You can leave off values and they will be passed over when the file is processed.
Do I need the codes or text responses in my source file?
Data Loader will recognize both text and code in your source files.
Can I map more than one incoming field to the same place in the database?
Yes, you can set up multiple fields to point to the same place in the database.
Does Data Loader create new events, behaviors, etc or do they need to be in the database first?
All behaviors, events, responses, etc need to be set up BEFORE you process the file in order to have them available in the mapping.
If you need new fields set up, please contact your audience services team.
My file has a an address column that mixes codes and names – what do I map it to?
If your columns have a mix of data you can map to the region or country (not the code options) and it will process correctly.
Will the “Update or Add Incoming” option blank out existing data if the current database field is not blank and the incoming field is blank?
No, the ‘update’ part of this will only work when there is incoming data present. If no data is present, nothing will write to the database.
TEMPLATES
Can I edit a template?
Yes, under the ‘Templates’ section you are able to edit any existing templates. The edits will apply to all files moving forward, but will not impact any files this template was previously applied to.
Is there a limit to the number of templates I can create?
You are able to create as many templates as you’d like for you and your team.
Is there a limit to the fields I can include in my template?
No, you are able to include as many fields as you’d like in your templates.
If I create a template from an existing file, what fields does it save to the template?
Everything in the existing mapping – including validation rules – carries over into the template when you use the “Create Template from Mapping” option.
Does the order of the mappings have to match the incoming column order for the template to apply? (e.g. if Email Address is listed first in my template, does that have to be my first column?)
No, the templates match on the column headers, not on the order of the headers. So long as the incoming file has the same header names, the template will apply.
What happens if my file has more/less columns than my template?
The template will apply the mapping to as many headers as it finds on the source file. If there are ‘extra’ columns in your source file that are not found in the template you will need to map them.
What happens if a template contains a demo/product/deployment type, etc that has been deactivated from my database?
The template piece that maps to a deactivated item will drop off of the template. You will either be left with an template set to ‘Active with Errors’ or, if it doesn’t impact the setup, the field will just be removed and the template will stay active.
VALIDATION RULES (SETTINGS)
Do I have to choose any of the rules?
Choosing validation rules is not required. However, it is strongly recommended that you choose at least one of the matching rules in order to avoid creating duplicates in your database.
Is there a limit to how many rules I can apply to my file?
No, you can choose multiple validation rules to apply to your file. The more rules you choose to apply, the cleaner the files and data will be in your database.
What rules are best for my file?
If you have any type of contact data, you will want to choose at least one matching rule.
For a break down of suggested rules for various types of files, check out ‘Validation Rules by Use Case‘
Can I prevent new people from being added to the database?
Yes, if you have a file and want to just update existing records, you can mark the rule “Reject New Customer”. No new records will be added.
If the incoming file has new people, can I add them to the database?
If, after applying matching rules, there are new names that can be added to the database, you can create them by choosing “Process Non-Match as Add”.
Do certain rules take priority over others?
Yes, when checked, the Customer Matching Lookup rules will run on the file first. If matches are now found, additional rules will be applied as you have them checked off.
POST PROCESSING
How can I find a file in Audience Builder?
There are 4 ways you can lookup a file in Audience Builder once it has processed:
- Tracking Code
- Keyword(s)
- Date Processed
- File Name
For full details on how to do this, check out our Querying in Audience Builder article.
Note: If you don’t see these lookup options in Audience Builder please ask your Audience Services team to configure them in for you.
Why is the number of records in Data Loader different from the total I’m seeing Audience Builder?
Audience Builder needs to be rebuilt in order to pull in new records. If you’ve processed a file with new names, they are not going to appear in Audience Builder automatically and will be added with the next rebuild.
If you need the new names displayed promptly, please contact your Audience Services team. If the count is still off, your Audience Services team can file an internal ticket to look into the counts.
Can I delete the file once I process it?
No, once a file has been processed you cannot delete it from the Data Loader interface. However, we do offer the option to Archive processed files in order to keep your list view clean. For more details on the archive process, check out the ‘Archiving Processed Files‘ section in our Loading Data article.
What happens if I process a file and made a mistake – Can it be undone?
During the processing stage, the incoming data from the source file is taken up and merged back into your database per the processing rules. In general, this process is irreversable.
If you have loaded a file and would like to explore ways to reset or repair mistakes please contact your Audience Services team promptly and they will work with you on a resolution.
Will I receive notification when my file has finished processing?
Yes, you will receive a confirmation email notifying you the file has finished processing. The email will include general stats on the number of rows input, number of rows processed and number of rows rejected. The email will also indicate the number of rows that created a new record and the number of rows that updated current records. If there are internal file duplicates in the input file, the first occurrence of the record will be counted as new record, all subsequent occurrences will be counted as update current records.
FAILURE TO PROCESS
Where can I see the reason why my file failed?
In the Actions drop-down on the list view you have an option to ‘View File State’. This will provide an explanation for why the file has failed.
If you need further explanation or direction, please contact your Audience Services team.
What row(s) caused the failure and why?
Once an error message has been returned, you can view the exact rows that failed and the explanation for the failure by going to ‘View File State’ from the Actions drop-down.
How can I correct the error(s) after a file has processed?
If the entire file has failed, you’ll need to re-upload the file, map it, and reprocess it.
If the file has completed with errors, you can manually pull the rows that errored from your source file, correct them, and upload them to process.
Note: A future enhancement will allow you to correct the errors from within the interface itself, but at this time it is not currently available and corrections must be done outside of the tool itself.
Why did my file fail?
The exact reason for your file failure can be found in the ‘View File State’. In general however, there tend to be 2 reasons why files fail:
1. Incorrect incoming values: When mapping any advanced fields (behaviors, demographics, products, or deployment types) the Incoming Value(s) provided must be an EXACT match to the source file. It is often the case that these values are not a match – especially when working with templates, and this causes a file to fail.
2. Gaps in validation rules: Failure to apply the correct validation rules in the settings step will often cause a file to fail. Misapplication of rules often relate to matching on existing records, or requiring fields (e.g. email address) that are not on the source file.
For additional insight into applying the correct validation rules, check out ‘Validation Rules by Use Case‘