Error/Issue:
Currently working on a pipeline which is copying data from a csv file and loading it into a database Table, so for that I configured the CopyData activity
i. Configured the source settings and to preview data provided path to a sample csv file
ii. Configured sink settings - provided the Table name and pre-copy scripts
iii. Once both have defined, next was the mapping between them and for that you have to click on the import schema, so what it does is basically gets the schema from this source file which you have provided and from database table you have mentioned in the sink side.
After it gets the schema from both the side, it presents you with the auto suggested links so if you find those ok, use that else you can change it as per your requirement .
That's what I did, but then when I started to test, got below error.
Why it happened
ErrorCode=UserErrorInvalidColumnMappingColumnCountMismatch,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=The column count in column mappings is more than the count in source/sink table.,Source=Microsoft.DataTransfer.ClientLibrary,'
As the error points out, the ADF runtime is telling you that the number of columns coming in the input file is less than the number of columns which are defined in the column mapping.
As can be seen in the above image the number of columns mapped is five but the file which is used for testing has less number of columns (in my case the input file had 3 columns)
What to do
The error can be of due to either reason
i. Incorrect column mapping
ii. Incorrect data in the file
In either of the cases you have to make sure that both are in adherence to each other. Here in my case the column mapping was correct and the input data file had issue - so corrected the data and issue was resolved.
Sometimes the issue can be with column mapping due to incorrect requirement understanding or use of incorrect sample file to generate schemas before mapping.
Knowledge Sharing is Caring !!!!!!
Learn More about some more Azure Data Factory errors
- Cannot find the object "xxx" because it does not exist or you do not have permissions.
- The function 'length' expects its parameter to be an array or a string. The provided value is of type 'Object'.
- ErrorCode=UserErrorInvalidColumnMappingColumnNotFound,Column 'xxx' specified in column mapping cannot be found in source data.
- The path 'xxx' specified in the add operation already exists. Please specify a new path
- Cannot create Sql Source. Error: The value of the property 'Value' is invalid for the stored procedure parameter 'XXX'
- Cannot connect to SQL Database:'XXX' . Verify that the instance name is correct
- Cannot open server 'xxx' requested by the login. Client with IP address 'x.x.x.x' is not allowed to access the server
- Operation on target Copy data failed: Failure happened on 'Source' side