Inconsistent Errors Using Bulk Insert With A Format File
May 16, 2006
As part of a c# program, utilizing .Net 2.0, I am calling a sproc via a SqlCommand to bulk load data from flat files to a various tables in a SQL Server 2005 database. We are using format files to do this, as all of the incoming flat files are fixed length. The sproc simply calls a T-SQL BULK INSERT statement, accepting the file name, format file name and the database table as input paramaters. As expected, this works most of the time, but periodically (to often for a production environment), the insert fails. The particular file to fail is essentially random and when I rerun the process, the insert completes successfully.
A sample of the error messages returned is as follows (@sql is the string executed):
Cannot bulk load. Invalid destination table column number for source column 1 in the format file "\RASDMNTTRAS_ROOTBCP_Format_FilesEMODT3.fmt".
Starting spRAS_BulkInsertData.
@sql = BULK INSERT Raser.dbo.EMODT3_Work FROM '\RASDMNTTRAS_ROOTAmeriHealthworkpdclmsemodt3.20060511.0915.txt.DATA' WITH (FORMATFILE = '\RASDMNTTRAS_ROOTBCP_Format_FilesEMODT3.fmt');
I want to use the bulk insert statement to insert data from a text file that contains more columns than the target sql table does. I am using SQL 7.0.
I am using a format file, but I can't work out how to achieve the above. SQL books online (and the msdn website) do not describe how to do this, but it is intimated that it can be done.
Hello, I am doing a bulk insert using a XML Format file from a csv file. Most everything works just fine, but my delimiter is a , the problem is one of the column sets sometimes contains a , within "" like this:
value1,"value,2",value3
So when I do my insert it is distorting the column values because unlike excel it is not ignoring the comma within the quotes. Is there any way to set an attribute within the format file to prevent this from happening?
I need to take only three coulmns from the file and the text file won't contain any delimeters between the fields. I tried to execute this using Bulk..Insert and i am getting the error:
Cannot bulk load. Invalid column number in the format file "C:sampleXXXX.fmt"
Any one can help me what is the problem here? I am using SQL Server 2000.
im attempting the following bulk insert statement: BULK INSERT gtaRatesTemp FROM 'D: estData.CSV' WITH (FIRSTROW = 3, FORMATFILE = 'D: estFormat.Fmt'
testData.CSV example:
"Country","Country Code","City","City Code","Currency","Item","Item Code","Address 1","Address 2","Address 3","Address 4","Telephone","Rating","Location 1","Location 2","Location 3","No. of","Dates","Dates","Days","Days","Min","Min","Basis","Twin","Single","Triple","Quad","Twin for sole use","Child","Child Age","Child Age","Meals included","Distance to City Centre (Kms)","Child Details","Special Conditions" "","","","","","","","","","","","","","","","","Rooms","From","To","From","To","Pax","Nights","","Price","Price","Price","Price","Price","Price","From","To","","","" "Australia","AA","Palm Cove, QLD","PALC","AUD(Australian Dollars)","ALASSIO ON THE BEACH(1BD POOL)","ALA","139 WILLIAMS ESPLANADE","PALM COVE","QUEENSLAND 4879","AUSTRALIA","61-7-40591550","4","Waterfront/Beach","","","24","14 Jun 2007","30 Jun 2007","Mon","Sun","1","2","Room","181.00","181.00","","","181.00","","","","Breakfast not included","1","","Cancellations: (14 Jun 2007-31 Mar 2008) 0-5 days prior 100% of total; 6-10 days prior 100% for 1 Night; 11 days or more No charge; " "Australia","AA","Palm Cove, QLD","PALC","AUD(Australian Dollars)","ALASSIO ON THE BEACH(1BD POOL)","ALA","139 WILLIAMS ESPLANADE","PALM COVE","QUEENSLAND 4879","AUSTRALIA","61-7-40591550","4","Waterfront/Beach","","","24","01 Jul 2007","31 Oct 2007","Mon","Sun","1","2","Room","231.00","231.00","","","231.00","","","","Breakfast not included","1","","Cancellations: (14 Jun 2007-31 Mar 2008) 0-5 days prior 100% of total; 6-10 days prior 100% for 1 Night; 11 days or more No charge; " "Australia","AA","Palm Cove, QLD","PALC","AUD(Australian Dollars)","ALASSIO ON THE BEACH(1BD POOL)","ALA","139 WILLIAMS ESPLANADE","PALM COVE","QUEENSLAND 4879","AUSTRALIA","61-7-40591550","4","Waterfront/Beach","","","24","01 Nov 2007","31 Mar 2008","Mon","Sun","1","2","Room","181.00","181.00","","","181.00","","","","Breakfast not included","1","","Cancellations: (14 Jun 2007-31 Mar 2008) 0-5 days prior 100% of total; 6-10 days prior 100% for 1 Night; 11 days or more No charge; " "Australia","AA","Palm Cove, QLD","PALC","AUD(Australian Dollars)","ANGSANA RESORT (1 BDRM BEACH)","ANG2","1 VEIVERS ROAD","PALM COVE","QUEENSLAND 4879","AUSTRALIA","61-7-40553000","5","Waterfront/Beach","","","67","14 Jun 2007","30 Jun 2007","Mon","Sun","1","1","Room","412.50","412.50","","","412.50","0","2","12","Breakfast not included","0","At this hotel an additional bed has not been provided in the room, child will need to share the existing bedding.","Cancellations: (14 Jun 2007-31 Mar 2008) 0-17 days prior 100% of total; 18-33 days prior 50% of total; 34 days or more No charge; " "Australia","AA","Palm Cove, QLD","PALC","AUD(Australian Dollars)","ANGSANA RESORT (1 BDRM BEACH)","ANG2","1 VEIVERS ROAD","PALM COVE","QUEENSLAND 4879","AUSTRALIA","61-7-40553000","5","Waterfront/Beach","","","67","01 Jul 2007","30 Nov 2007","Mon","Sun","1","1","Room","463.00","463.00","","","463.00","0","2","12","Breakfast not included","0","At this hotel an additional bed has not been provided in the room, child will need to share the existing bedding.","Cancellations: (14 Jun 2007-31 Mar 2008) 0-17 days prior 100% of total; 18-33 days prior 50% of total; 34 days or more No charge; " testFormat.Fmt:
Similar to a previous post (http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=244646&SiteID=1), I am trying to import data into a SQL Table.
I am trying to program a small application that will import product data obtained through suppliers via CD-ROM. One supplier in particular uses Fixed width colums, and data looks like this:
Example of Data
0124015Apple Crate 32.12
0124016Bananna Box 12.56
0124017Mango Carton 15.98
0124018Seedless Watermelon 42.98 My Table would then have: ProductID as int Name as text Cost as money
How would I go about extracting the data with an XML Format file? I am stumbling over how to tell it where to start picking up data for a specific column. Is there any way that I could trim the Name column (i.e.: "Mango Carton " --> "Mango Carton")?
I don't know if it makes any difference, but I've been calling SQL from my code by doing this:
Code in C# Form
SqlConnection SqlConnection = new SqlConnection(global::SQLClients.Properties.Settings.Default.ClientPhonebookConnectionString); SqlCommand cmd = new SqlCommand();
SqlConnection.Open(); cmd.ExecuteNonQuery(); SqlConnection.Close(); RefreshData(); I am running Visual Studio C# Express 2005 and SQL Server Express 2005.
I'm trying to use Bulk insert for the first time and getting the following error. I think it might have something to do with my Format File and from the error msg there's a conversion error for the first column. In my database the Field is nvarchar(6) so my best guess is to use SQLNChar for the first column. I've checked the end of each line is CR LF therefore the is correct for line 7 right?
Msg 4863, Level 16, State 1, Line 1 Bulk load data conversion error (truncation) for row 1, column 1 (ASXCode). Msg 7399, Level 16, State 1, Line 1 The OLE DB provider "BULK" for linked server "(null)" reported an error. The provider did not give any information about the error. Msg 7330, Level 16, State 2, Line 1 Cannot fetch a row from OLE DB provider "BULK" for linked server "(null)".
BULK INSERTtbl_ASX_Data_temp FROM 'M:DataASXImportTest.txt' WITH (FORMATFILE='M:DataASXSQLFormatImport.Fmt')
Msg 4862, Level 16, State 1, Server PATH-SQLDEV, Line 2 Cannot bulk load because the file "c:DATABATCHBCPFormat.fmt" could not be read. Operating system error code (null).
Above is the error I get. The problem is I do not know what is causing this error. It occurs when I attempt to use SQLCMD with bulk insert.
I am using SQLserver 2005 and I have a similar set up in a test database that works, why this format file does not is beyond me, but my experience is when the format file has an error in it, such as a mispelled datatype or a incorrect column number, the error will zero in on that, rather then declare the entire file unreadable. Furthermore if I go into the file, change something so where it is incorrect (like a column number) it will zero in on that error. So I know that the format file can be read. If I knew what this error was all about I would at least know where to begin in fixing it. I have also tried using a very small sample file for the data being inserted. Same error.
I have to perform a bulk Import on a regular Basis and have created a script to do this. The Problem is that the .csv file has 12 Columns and the table to Import into has 14. To Workaround this discrepancy I have decided to use a Format file. The Problem is that how to create one.
With "bcp MyDatabase.dbo.MyTable out C:MyFile.Dat -n -T" command line, I could get an exported data file. And I can also import this file into MyTable using 'BULK INSERT MyDatabase.dbo.MyTable FROM 'C:MyFile.dat' WITH (DATAFILETYPE='native');' query statement.
Now, I want to make my own data file just like made by bcp above. Although I could make file of 'char' type, 'native' type file is needed for performance and other reasons. And the format file should not be used.
I are using a BULK INSERT to insert the data from a ascii file to a sql table. The table has a ProductInstanceId column that exists in the tables but does not exist in the ascii DICast data. I am setting the ProductInstanceId to a Guid that will be used for Metrics. I would like to create the Guid in C++ and then set it somehow during the BULK INSERT DICastRaw1hr and DICastRaw6hr. I am calling the BULK INSERT from C++/ADO. I do not see how you can set a static data in the BULK INSERT for a column that exists in the table but does not the source data ... seems there should be a way to do this with the format file?
The other way to do this is with a TRIGGER. I have the TRIGGER below. Prior to the calling the BULK INSERT using ADO I will use ADO to ALTER the TRIGGER with the new Guid. When the BULK INSERT runs the ProductInstanceId will be populated with the new Guid.
ALTER TRIGGER DICastRaw1hrInsertGuid ON Alphanumericdata.dbo.DICastRaw1hr FOR INSERT AS UPDATE dbo.DICastRaw1hr SET ProductInstanceId = '4f9a44eb-092b-445b-a224-cc7cdd207092' WHERE modelrundatetime = (select max(modelrundatetime) from Alphanumericdata.dbo.DICastraw1hr(NOLOCK))
More Questions:
- The Trigger is slow. The Bulk Insert without the Trigger runs in about 10 sec ... with the Trigger in about 40 sec. I tried to use the sql code below in the TRigger but it was only doing the UPDATE on the last row. The TRIGGER must run after the BULK INSERT is complete. Now I am using the select (bad). Any comments ...
ALTER TRIGGER DICastRaw1hrInsertDate ON Alphanumericdata.dbo.DICastRaw1hr FOR INSERT AS DECLARE @ID as integer SELECT @ID = i.recordid from inserted i UPDATE dbo.DICastRaw1hr SET ProductInstanceId = '4f9a44eb-092b-445b-a224-cc7cdd207092' WHERE recordid = @ID
- I understand that I could set the Guid in the Default Value part of the table definition using the NEWID() function. I need the Guid to be the same for all the rows that are inserted during the BULK INSERT (all have the same modelrundatetime) ... how would I do this?
I am trying to load a fixed width text file using `Bulk Insert` and a XML format file. I have used the same process and XML file on another fixed width, except with less columns.
Error Msg 4857, Level 16, State 1, Line 16 Line 4 in format file "PATHCaddr.xml": Attribute "type" could not be specified for this type.
My colleague is working on bulk insert task from SSIS and since the data file does not contain any valid delimeter one of the suggestion he got is to use a file format to address the issue. Thus a bcp command is used to generate the format file, as per below.
The file file format was generated, from the data flow we added the BULK INSERT task and set the properties accordingly including the File Format and location of the file. Upon running the task itself we encountered the error as per below.
[Bulk Insert Task] Error: An error occurred with the following error message: "Cannot bulk load. The file "C:HFISTAT.fmt" does not exist.".
Progress: The Bulk Insert task is completed. - 100 percent complete
Task Bulk Insert Task failed
Have checked the file and it is in C: drive and it is not protected or read-only. Validated the output file and it is as per expected. Any help would be appreciated very much.
ISSUE: ==================== In SQL 2005 (sp2) I get the following error when preforming a bulk insert with an associated xml format file: "Could not bulk insert. Unknown version of format file"
Question: ==================== I am unsure what they mean by "unknown version". Specifically the format file in question was created using bcp. Also the entire table scenario was created from a msdn example.
Any ideas? have you seen this before?
NOTE: i can reproduce this issue outside the example but will refer to msdn considering it is simple and easily reproducible.
Scenario ==================== I can reproduce this error with the BULK INSERT example discussed on msdn (example A) http://msdn2.microsoft.com/en-us/library/ms191234.aspx
TO REPRODUCE:
* In short the table structure is: Person (Age int, FirstName varchar(20), LastName varchar(30))
* Data File Template: Age<tab>Firstname<tab>Lastname<return>
* Here is some the actual sql statement that pulls this all together BULK INSERT mytestnames FROM 'C:datatestexampledata-c.Dat' WITH (FORMATFILE = 'C:datatestexamplefmt.Fmt');
ISSUE: ==================== In SQL 2005 (sp2) I get the following error when preforming a bulk insert with an associated xml format file: "Could not bulk insert. Unknown version of format file"
Question: ==================== I am unsure what they mean by "unknown version". Specifically the format file in question was created using bcp. Also the entire table scenario was created from a msdn example.
Any ideas? have you seen this before?
NOTE: i can reproduce this issue outside the example but will refer to msdn considering it is simple and easily reproducible.
Scenario ==================== I can reproduce this error with the BULK INSERT example discussed on msdn (example A) http://msdn2.microsoft.com/en-us/library/ms191234.aspx
TO REPRODUCE:
* In short the table structure is: Person (Age int, FirstName varchar(20), LastName varchar(30))
* Data File Template: Age<tab>Firstname<tab>Lastname<return>
* Here is some the actual sql statement that pulls this all together BULK INSERT mytestnames FROM 'C:datatestexampledata-c.Dat' WITH (FORMATFILE = 'C:datatestexamplefmt.Fmt');
I am getting inconsistent results when BULK INSERTING data from a tab-delimited text file. As part of my testing, I run the same code on the same file again and again, and I get different results every time! I get this on SQL 2005 and SQL 2012 R2.We have an application that imports data from a spreadsheet. The sheet contains section headers with account numbers and detail rows with transactions by date:
AAAA.1234 /* (account number)*/ 1/1/2015 $150 First Transaction 1/3/2015 $24.233 Second Transaction
BBBB.5678 1/1/2015 $350 Third Transaction 1/3/2015 $24.233 Fourth Transaction
My Import program saves this spreadsheet at tab-delimited text, then I use BULK INSERT to bring the data into a generic table full of varchar(255) fields. There are about 90,000 rows in each day's data; after the BULK INSERT about half of them are removed for various reasons. Next I add a RowID column to the table with the IDENTITY (1,1) property. This gives my raw data unique row numbers.
I then run a routine that converts and copies those records into another holding table that's a copy of the final destination table. That routine parses though the data, assigning the account number in the section header to each detail row. It ends up looking like this:
AAAA.1234 1/1/2015 $150 First Purchase AAAA.1234 1/3/2015 $24.233 Second Purchase BBBB.5678 1/1/2015 $350 Third Purchase BBBB.5678 1/3/2015 $24.233 Fourth Purchase
My technique: I use a cursor to get the starting RowID for each Account Number: I then use the upper and lower RowIDs to do an INSERT into the final table. The query looks like this:
SELECT RowID, SUBSTRING(RowHeader, 6,4) + '.UBC1' AS AccountNumber FROM GenericTable WHERE RowHeader LIKE '____.____%'
Results look like this:
But every time I run the routine, I get different numbers! my results are not accurate. I get inconsistent results EVERY TIME.Here is my code, with table, field and account names changed for business confidentiality.This is a high profile project at my company;
TRUNCATE TABLE GenericImportTable; ALTER TABLE GenericImportTable DROP COLUMN RowID; BULK INSERT GenericImportTable FROM 'SERVERGeneralAppnameDataFile.2015.05.04.tab.txt' WITH (FIELDTERMINATOR = ' ', ROWTERMINATOR = ' ', FIRSTROW = 6)
I'm trying to set up a BULK INSERT Format File for some data that I've been sent, which, according to the data documentation, comes in fixed-width format fields (no delimiters except for end-of-row 0D0A) in SQL-Server 2005 Express.
The following is the first line... "7999163 09182003 56586 56477 3601942 1278 22139 1102 113 118 51450 1 1 63535647 10000 7999162 09182003 56586 56477 3601942 1279 22139 1102 113 118 51450 1 1 63535647 10000 "
Looking with a hex editor, all the above whitespace are 20's.
From the documentation, I've constructed the following table...
However... actually running this gives the following error...
Msg 4863, Level 16, State 4, Line 1 Bulk load data conversion error (truncation) for row 1, column 13 (EXISTENCE). Msg 7399, Level 16, State 1, Line 1 The OLE DB provider "BULK" for linked server "(null)" reported an error. The provider did not give any information about the error. Msg 7330, Level 16, State 2, Line 1 Cannot fetch a row from OLE DB provider "BULK" for linked server "(null)".
Since this is my first time with this, I read the BOL items on Bulk Insert, Format Files, and each of the formatting attibutes, and made up two line "toy" examples for SQLCHAR and SQLINT, including two columns - all worked as expected.
It seemed that only SQLNUMERIC/SQLDECIMAL fell apart.
Even the following trivial example doesn't work for this field of data...
Msg 9803, Level 16, State 1, Line 1 Invalid data for type "numeric". The statement has been terminated.
or
9.0 1 1 SQLNUMERIC 0 10 "/r/n" 1 TRACER_ID ""
which gives this error...
Msg 4832, Level 16, State 1, Line 1 Bulk load: An unexpected end of file was encountered in the data file. Msg 7399, Level 16, State 1, Line 1 The OLE DB provider "BULK" for linked server "(null)" reported an error. The provider did not give any information about the error. Msg 7330, Level 16, State 2, Line 1 Cannot fetch a row from OLE DB provider "BULK" for linked server "(null)".
Also - there was the DB_CREATE_DATE and DB_UPDATED_DATE CHAR (8) were supposed to be dates in the format of mmddyyy but clearly there is no Date datatype in SQL-Server. I would suppose these need to be converted, but am unsure how. What is clear is that the data was dumped from Oracle in text form,
Any thoughts on this would be greatly appreciated...
I'm trying to set up a BULK INSERT Format File for some data that I've been sent, which, according to the data documentation, comes in fixed-width format fields (no delimiters except for end-of-row 0D0A) in SQL-Server 2005 Express.
The following is the first line... "7999163 09182003 56586 56477 3601942 1278 22139 1102 113 118 51450 1 1 63535647 10000 7999162 09182003 56586 56477 3601942 1279 22139 1102 113 118 51450 1 1 63535647 10000 "
Looking with a hex editor, all the above whitespace are 20's.
From the documentation, I've constructed the following table...
However... actually running this gives the following error...
Msg 4863, Level 16, State 4, Line 1 Bulk load data conversion error (truncation) for row 1, column 13 (EXISTENCE). Msg 7399, Level 16, State 1, Line 1 The OLE DB provider "BULK" for linked server "(null)" reported an error. The provider did not give any information about the error. Msg 7330, Level 16, State 2, Line 1 Cannot fetch a row from OLE DB provider "BULK" for linked server "(null)".
Since this is my first time with this, I read the BOL items on Bulk Insert, Format Files, and each of the formatting attibutes, and made up two line "toy" examples for SQLCHAR and SQLINT, including two columns - all worked as expected.
It seemed that only SQLNUMERIC/SQLDECIMAL fell apart.
Even the following trivial example doesn't work for this field of data...
Msg 9803, Level 16, State 1, Line 1 Invalid data for type "numeric". The statement has been terminated.
or
9.0 1 1 SQLNUMERIC 0 10 "/r/n" 1 TRACER_ID ""
which gives this error...
Msg 4832, Level 16, State 1, Line 1 Bulk load: An unexpected end of file was encountered in the data file. Msg 7399, Level 16, State 1, Line 1 The OLE DB provider "BULK" for linked server "(null)" reported an error. The provider did not give any information about the error. Msg 7330, Level 16, State 2, Line 1 Cannot fetch a row from OLE DB provider "BULK" for linked server "(null)".
Also - the DB_CREATE_DATE and DB_UPDATED_DATE CHAR (8) were supposed to be dates in the format of mmddyyy but clearly there is no Date datatype in SQL-Server. I would suppose these need to be converted, but am unsure how. What is clear is that the data was dumped from Oracle in text form,
Any thoughts on this would be greatly appreciated...
I'm trying to set up a BULK INSERT Format File for some data that I've been sent, which, according to the data documentation, comes in fixed-width format fields (no delimiters except for end-of-row 0D0A) in SQL-Server 2005 Express.
The following is the first line... "7999163 09182003 56586 56477 3601942 1278 22139 1102 113 118 51450 1 1 63535647 10000 7999162 09182003 56586 56477 3601942 1279 22139 1102 113 118 51450 1 1 63535647 10000 "
Looking with a hex editor, all the above whitespace are 20's.
From the documentation, I've constructed the following table...
However... actually running this gives the following error...
Msg 4863, Level 16, State 4, Line 1 Bulk load data conversion error (truncation) for row 1, column 13 (EXISTENCE). Msg 7399, Level 16, State 1, Line 1 The OLE DB provider "BULK" for linked server "(null)" reported an error. The provider did not give any information about the error. Msg 7330, Level 16, State 2, Line 1 Cannot fetch a row from OLE DB provider "BULK" for linked server "(null)".
Since this is my first time with this, I read the BOL items on Bulk Insert, Format Files, and each of the formatting attibutes, and made up two line "toy" examples for SQLCHAR and SQLINT, including two columns - all worked as expected.
It seemed that only SQLNUMERIC/SQLDECIMAL fell apart.
Even the following trivial example doesn't work for this field of data...
Msg 9803, Level 16, State 1, Line 1 Invalid data for type "numeric". The statement has been terminated.
or
9.0 1 1 SQLNUMERIC 0 10 "/r/n" 1 TRACER_ID ""
which gives this error...
Msg 4832, Level 16, State 1, Line 1 Bulk load: An unexpected end of file was encountered in the data file. Msg 7399, Level 16, State 1, Line 1 The OLE DB provider "BULK" for linked server "(null)" reported an error. The provider did not give any information about the error. Msg 7330, Level 16, State 2, Line 1 Cannot fetch a row from OLE DB provider "BULK" for linked server "(null)".
Also - the DB_CREATE_DATE and DB_UPDATED_DATE CHAR (8) were supposed to be dates in the format of mmddyyy but clearly there is no Date datatype in SQL-Server. I would suppose these need to be converted, but am unsure how. What is clear is that the data was dumped from Oracle in text form,
Any thoughts on this would be greatly appreciated...
I try to import data with bulk insert. Here is my table:
CREATE TABLE [data].[example]( col1 [varchar](10) NOT NULL, col2 [datetime] NOT NULL, col3 [date] NOT NULL, col4 [varchar](6) NOT NULL, col5 [varchar](3) NOT NULL,
The first column should store double (in col2 and col3) in my table
My file: Col1,Col2,Col3,Col4,Col5,Col6,Col7 2015-04-30@|@MDDS@|@ADP@|@EUR@|@185.630624@|@2015-04-30@|@MDDS 2015-04-30@|@MDDS@|@AED@|@EUR@|@4.107276@|@2015-04-30@|@MDDS
My command: bulk insert data.example from 'R:epoolexample.csv' WITH(FORMATFILE = 'R:cfgexample.fmt' , FIRSTROW = 2)
Get error: Msg 4823, Level 16, State 1, Line 2 Cannot bulk load. Invalid column number in the format file "R:cfgexample.fmt".
I changed some things as: used ";" and "," as column delimiter changed file type from UNIX to DOS and adjusted the format file with " " for row delimiter
Removed this line from format file 1 SQLCHAR 0 10 "@|@" 2 Col2 "" Nothing works ....
I have a stored procedure that dynamically bulk loads several tables from several text files. If I encounter an error bulk loading a table in the stored procedure, all I get is the last error code produced, but if I run the actual bulk load commands through SQL Management Studio, it gives much more usable errors, which can include the column that failed to load. We have tables that exceed 150 columns (don't ask), and having this information cuts troubleshooting load errors from hours down to minutes. Onto my question..., is there any way to capture all of the errors produced by the bulk load from within a stored procedure (see examples below)?
Running this...
BULK INSERT Customers
FROM 'c: estcustomers.txt'
WITH (TabLock, MaxErrors = 0, ErrorFile = 'c: estcustomers.txt.err')
Produces this (notice column name at the end of the first error)...
Msg 4864, Level 16, State 1, Line 1
Bulk load data conversion error (type mismatch or invalid character for the specified codepage) for row 1, column 1 (CustId).
Msg 7399, Level 16, State 1, Line 1
The OLE DB provider "BULK" for linked server "(null)" reported an error. The provider did not give any information about the error.
Msg 7330, Level 16, State 2, Line 1
Cannot fetch a row from OLE DB provider "BULK" for linked server "(null)".
Running this (similar to code in my stored procedure)...
BEGIN TRY
BULK INSERT Customers
FROM 'c: estcustomers.txt'
WITH (TabLock, MaxErrors = 0, ErrorFile = 'c: estcustomers.txt.err')
END TRY
BEGIN CATCH
SELECT
ERROR_NUMBER() AS ErrorNumber,
ERROR_SEVERITY() AS ErrorSeverity,
ERROR_STATE() AS ErrorState,
ERROR_PROCEDURE() AS ErrorProcedure,
ERROR_LINE() AS ErrorLine,
ERROR_MESSAGE() AS ErrorMessage;
END CATCH
Produces something similar to this (which is useless)... ...Cannot fetch a row from OLE DB provider "BULK" for linked server "(null)".
I used bcp to produce the apended format file.How can it be modified to recognize the quotes that surround the textfields and not insert the quotes along with the text? Invariably, thefirst four columns have text surrounded by quotes and are terminated bytabs. If the first column has "abc", only abc ought to be insertedinto that field in the table.ThanksTed==================format file========================<?xml version="1.0" ?>- <BCPFORMATxmlns="http://schemas.microsoft.com/sqlserver/2004/bulkload/format"xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">- <RECORD><FIELD ID="1" xsi:type="CharPrefix" PREFIX_LENGTH="2" MAX_LENGTH="6"COLLATION="Latin1_General_CI_AI" /><FIELD ID="2" xsi:type="CharPrefix" PREFIX_LENGTH="2" MAX_LENGTH="7"COLLATION="Latin1_General_CI_AI" /><FIELD ID="3" xsi:type="CharPrefix" PREFIX_LENGTH="2" MAX_LENGTH="48"COLLATION="Latin1_General_CI_AI" /><FIELD ID="4" xsi:type="NativeFixed" LENGTH="4" /><FIELD ID="5" xsi:type="CharPrefix" PREFIX_LENGTH="2" MAX_LENGTH="8"COLLATION="Latin1_General_CI_AI" /><FIELD ID="6" xsi:type="NativePrefix" PREFIX_LENGTH="1" /></RECORD>- <ROW><COLUMN SOURCE="1" NAME="f_supplier_code" xsi:type="SQLVARYCHAR" /><COLUMN SOURCE="2" NAME="f_product_code" xsi:type="SQLVARYCHAR" /><COLUMN SOURCE="3" NAME="f_product_name" xsi:type="SQLVARYCHAR" /><COLUMN SOURCE="4" NAME="f_asset_classes_id" xsi:type="SQLINT" /><COLUMN SOURCE="5" NAME="f_size" xsi:type="SQLVARYCHAR" /><COLUMN SOURCE="6" NAME="f_dist_unit" xsi:type="SQLFLT8" /></ROW></BCPFORMAT>
I got a few *.csv files which I have to import in a table. The Insert looks like this:
bulk insert actocashdb.dbo.MyTable from 'c:UpdatesPLUArtikel_01.csv' with ( fieldterminator = ';', DATAFILETYPE = 'char', tablock, codepage = 'ACP' )
There is a field which contains a Date and Time in US-Format (e.g. 08/15/2007 03:15:00 PM). Some of my SQL - Servers (in fact they are just MSDE) are configured as German language. So, when I try to Insert the File I get a Type Mismatch Error, because the MSDE awaits the Date in German Format (e.g. 15.08.2007 15:15:00). Are there any options somewhere so I can Insert this Date? NOTE: It is not possible to reconfigure the MSDE to English.
I'm in the process of trying to cutover SLQ 2000 database activity to a newer used server running Windows 2003. All the latest Win & SQL Service Packs through late Feb 2007 have been installed. The server has a Promise Fast Track S150 TX2plus SCSI raid with 2 80 gb drives. I have been using at as database development system for 3 months without any issues.
This morning I restored the latest full backup and 2 intervening transaction log backups. I am the only user on the system today. I ran DBDCC CHECKDB WITH TABLERESULTS 5 times over a 45 minute period and received 5 different results!
Run 1 - No Error
Run 2 - Following Errors:
2511162Table error: Object ID 754101727, Index ID 0. Keys out of order on page (1:631938), slots 40 and 41.repair_rebuild
8990101CHECKDB found 0 allocation errors and 1 consistency errors in table 'DatabaseEventLog' (object ID 754101727).
8928161Object ID 1218103380, index ID 255: Page (1:373888) could not be processed. See other errors for details.repair_allow_data_loss
8990101CHECKDB found 0 allocation errors and 1 consistency errors in table 'WeatherTransaction' (object ID 1218103380).
Run 3 - Following Errors:
2511162Table error: Object ID 754101727, Index ID 0. Keys out of order on page (1:631938), slots 40 and 41.repair_rebuild
8990101CHECKDB found 0 allocation errors and 1 consistency errors in table 'DatabaseEventLog' (object ID 754101727).
8928161Object ID 1218103380, index ID 255: Page (1:373888) could not be processed. See other errors for details.repair_allow_data_loss
8990101CHECKDB found 0 allocation errors and 1 consistency errors in table 'WeatherTransaction' (object ID 1218103380).
Run 4 - Following Errors:
2511162Table error: Object ID 754101727, Index ID 0. Keys out of order on page (1:631938), slots 40 and 41.repair_rebuild
8990101CHECKDB found 0 allocation errors and 1 consistency errors in table 'DatabaseEventLog' (object ID 754101727).
8928161Object ID 1218103380, index ID 255: Page (1:373888) could not be processed. See other errors for details.repair_allow_data_loss
894116102Table error: Object ID 1218103380, index ID 255, page (1:373888). Test (sorted [i].offset >= PAGEHEADSIZE) failed. Slot 8, offset 0x2 is invalid.repair_allow_data_loss
894216101Table error: Object ID 1218103380, index ID 255, page (1:373888). Test (sorted[i].offset >= max) failed. Slot 0, offset 0x8 overlaps with the prior row.repair_allow_data_loss
8990101CHECKDB found 0 allocation errors and 3 consistency errors in table 'WeatherTransaction' (object ID 1218103380).
Run 5 - No Error!
I ran several of the obligatory CHKDSK and had no errors:
c:>chkdsk The type of the file system is NTFS.
WARNING! F parameter not specified. Running CHKDSK in read-only mode.
CHKDSK is verifying files (stage 1 of 3)... File verification completed. CHKDSK is verifying indexes (stage 2 of 3)... Index verification completed. CHKDSK is verifying security descriptors (stage 3 of 3)... Security descriptor verification completed.
78116030 KB total disk space. 45291176 KB in 33231 files. 10496 KB in 3135 indexes. 0 KB in bad sectors. 121086 KB in use by the system. 65536 KB occupied by the log file. 32693272 KB available on disk.
4096 bytes in each allocation unit. 19529007 total allocation units on disk. 8173318 allocation units available on disk.
Are these DBDCC CHECKDB WTIH TABLERESULTS differences normal or is this a bug or hardware issue? Any advice you can offer would be appreciated.
I have a small problem - I am unable to load data from a .csv file into a table in SQL Server. Here is the command I am running: BULK INSERT CCSProgramParticipation FROM 'c: est.csv' WITH ( DATAFILETYPE = 'char', FIELDTERMINATOR = ',', ROWTERMINATOR = '' )
Data in test.csv is the following format: (date fields can be blank) NY580232,0,6/30/2006,3567,396,7/1/2005,9/9/2005 NY580232,0,6/30/2006,14850,462,12/12/2005, .... ....
What I see is the data does get loaded; however, data from the following row is getting inserted in the last field of a particular row (previous row) - it seems like the rowterminator is being ignored.
Has anyone encountered this issue? Please let me know your thoughts on this.
BULK INSERT testTable FROM 'C:UsersRobsDocumentsSoccer2011-2012SC1.csv' WITH ( FIRSTROW=2, FIELDTERMINATOR = ',', ROWTERMINATOR = ' ', )
Works fine. Most of my csv files have the same number of columns, but some have 4 less. The files contain the same column headings as the full size ones(only 4 less). Is there a way when bulk inserting for sql to either skip these, or ignore the error.
Hi there,I have some text files saved using a UTF-8 encoding. The "BULK INSERT"statment in Sql Server 2000 is failing with a column length error. Savingthe file as ASCII removes the problem. However, I would like to import filesin UTF-8 format. I understand that the BCP tool, when used from the commandline, can take an "-F UTF8" argument, which allows it to work. Can this bedone from the SQL> prompt with the BULK INSERT statement?Cheers,Tobin
hi friends i am using bulk insert command for txt files but now i want to use bulk insert command for dbf files so plz any one can tell me how to use this command for dbf files thanx in advance.........
