HdfsBridge::recordReaderFillBuffer - Unexpected error encountered filling record reader buffer: MalformedInputException: Input length = 1

byMaheshkumar Tiwari •
1
{tocify} $title={Table of Contents}

Error/Issue:


Working on Polybase implementation where data stored as parquet file has to be imported in Synapse (SQL DWH). 

So based on number of objects, created respective polybase(s). After executing few of them successfully, I got below error for one of them


HdfsBridge::recordReaderFillBuffer - Unexpected error encountered filling record reader buffer: MalformedInputException: Input length = 1


To troubleshoot  checked the error statement ---

HdfsBridge::recordReaderFillBuffer - Unexpected error encountered filling record reader buffer: MalformedInputException: Input length = 2 


It appeared something in Input has length = 2, (size mismatch) but upon checking file found no such fields.

Why it happened


I have created the external table with varchar as the datatype for all the columns so that all datatypes can be handled.

             CREATE EXTERNAL TABLE SampleTbl(
ID VARCHAR(200),
                        NOTE VARCHAR(1000),
ROWGUID VARCHAR(200),
CreateDate VARCHAR(200)
             )


But there's an exception, columns having datetime values and that's the reason for the error.  

Although external table will be created, the error will come while inserting in destination table, where expected is datetime value but trying to be inserted is varchar.

The error message doesn't say so 😉


What to do


So while creating external table,  check for columns having datetime datatype. If there is then specify it .

            CREATE EXTERNAL TABLE SampleTbl(
ID VARCHAR(200),
                        NOTE VARCHAR(1000),
ROWGUID VARCHAR(200),
CreateDate DATETIME
            )






If you have questions or suggestions, feel free to do in comments section below !!!


Do share if you find this helpful .......
 
                          Knowledge Sharing is Caring !!!!!!






Related Post 


    Tags:

    1 Comments

    If you have any suggestions or questions or want to share something then please drop a comment

    1. AnonymousJune 9, 2022 at 12:51 PM

      Msg 107090, Level 16, State 1, Line 36
      HdfsBridge::recordReaderFillBuffer - Unexpected error encountered filling record reader buffer: HadoopExecutionException: Too long string in column [-1]: Actual len = [21]. MaxLEN=[20]

      ReplyDelete
    Previous Post Next Post