Converting Flat File To SQL2005 Table (Flat File From H***)

Feb 11, 2008

First, a couple of important bits of information. Until last week, I had never touched SISS, and therefore, I know very little about it. I just never had the need to use it...until now. I was able to convert my first 3 flat files to SQL2005 tables by right clicking on "SISS Package" and choosing "SISS Import and Export Wizard". That is the extent of my knowledge! So please, please, please be patient with me and be as descriptive as possible.

I thought I could attach some sample files to this post, but it doesn't look like I can. I'll just paste the information below in two separate code boxes. The first code box is the flat file specifications and the second one is a sample single line flat file similar to what I'm dealing with (the real flat file is over 2 gigs).

My questions are below the sample files.


Code Snippet
Record Length 400

Positions Length FieldName

Record Type 01
1,2 L=2 Record Type (Always "01")
3,12 L=10 Site Name
13,19 L=7 Account Number
20,29 L=10 Sub Account
30,35 L=6 Balance
36,37 L=1 Active
37,41 L=5 Filler
Record Type 02
1,2 L=2 Record Type (Always "02")
3,4 L=2 State
5,30 L=26 Address
31,41 L=11 Filler
Record Type 03
1,2 L=2 Record Type (Always "03")
3,6 L=4 Coder
7,20 L=14 Locator ID
21,22 L=2 Age
23,41 L=19 Filler
Record Type 04
1,2 L=2 Record Type (Always "04")
3,9 L=7 Process
10,19 L=10 Client
20,26 L=6 DOB
26,41 L=16 Filler
Record Type 05
1,2 L=2 Record Type (Always "05")
3,16 L=14 Guarantor
17,22 L=6 Guar Account
23,23 L=1 Active Guar
**There can be multiple 05 records, one for each Guarantor on the account**


and the single line flat file...



Code Snippet
01Site1 12345 0000098765 Y 02NY1155 12th Street 03ELL 0522071678 29 04TestingSmith,Paul071678 05Smith, Jane 445978N 05Smith, Julie 445989N 05Smith, Jenny 445915N 01Site2 12346 0000098766 N 02MN615 Woodland Ct 04InfoJones,Chris 012001 01Site3 12347 0000098767 Y 02IN89 Jade Street 03OWB 6429051282 25 04Screen New,Katie 879500





As you can see, each entry could have any number of records and multiples of some of the record types, with one exception, every entry must have a "01" record and can only have one "01" record. Oh, and each record has a length of 400.

I need to get this information into a SQL 2005 database so I can create a front end for accessing the data. Originally, I wanted one line for each account and have null values listed for entries that don't have a specific record. Now that I've looked at the data again, that doesn't look like a good idea. I think a better way to do it would be to create 5 different tables, one for each record type. However, records 2 through 5 don't have anything I can make a primary key. So here are my questions...


Is it possible to make 5 tables from this one file, one table for each of the record types?

If so, can I copy the Account number in record 01, position 13-19 in each of the subsequent record types (that way I could link the tables as needed)?

Can this be done using the SISS Import and Export Wizard to create the package? If not, I'm going to need some very basic step by step instructions on how to create the package.

Is SISS the best way to do this conversion or is there another program that would be better to use?
I know this is a huge question and I appreciate the help of anyone who boldly decides to help me! Thank you in advance and I welcome anyone's suggestions!

View 13 Replies


ADVERTISEMENT

Unable To Edit Pre-defined Flat File Connection Manager Properties In The Flat File Destination Editor

Aug 24, 2007

Hi,

I am testing SSIS and have created a Flat File Destination. I defined the Flat File Connection as New for the first time and it worked fine. Now, I would like to go back and modify the Flat File Connection in the Flat File Destination Editor, but it allows only to create a New connection rather allowing me to edit the existing one. For testing, I can go back and create a new connection, but if my connection had 50-100 columns then it would be an issue to re-create it from scratch.

Did someone else faced this issue?


Thanks,
AQ

View 1 Replies View Related

Flat File Connection Manager Throws Error When A Column Gets Added To The Flat File

Dec 27, 2006

Hi,

I have a situation where a tab limited text file is used to populate a sql server table.

The tab limited text file comes from a third party vendor. There are fixed number of columns we need to export to the sql server table. However the third party may add colums in the text file. Whenenver the text file has an added column (which we dont need to import) the build fails since the flat file connection manager does not create the metadata for it again. The problem goes away  where I press the button "Reset Columns" since it builds the metadata then. Since we need to build the tables everyday we cannot automate it using SSIS because the metadata does not change automatically. Is there a way out in SSIS?

View 5 Replies View Related

Output Column Width Not Refected In The Flat File That Is Created Using A Flat File Destination?

May 11, 2006

I am transferring data from an OLEDB source to a Flat File Destination and I want the column width for all of the output columns to 30 (max width amongst the columns selected), but that is not refected in the Fixed Width Flat File that got created. The outputcolumnwidth seems to be the same as the inputcolumnwidth. Is there any other setting that I am possibly missing or is this a possible defect?

Any inputs will be appreciated.

M.Shah

View 3 Replies View Related

How To Redirect The Error Of A Source Flat File To The Destination Flat File?

Nov 10, 2006

Hi all,

I m using SSIS and i am transfering the data from Flat File Source to the OLE DB destination File. The source file contain some corrupt data which i am transfering to the other Flat file destination file.

Debugging is succesful but i am not getting any error output in the Flat file destination file.

i had done exactly which is written in the msdn tutorial of SSIS.

Plz tell me why i am not getting the error output in the destination flat file?

thanx

View 1 Replies View Related

How Can I Take This Example Flat File And Parse Out Each Section To A New Flat File? Each Section Starts With HD (header Row)

Mar 13, 2006

How can I take this example Flat file and parse out each section to a new flat file?  Each section starts with HD (header row)

http://www.webfound.net/flat_file_example.txt

e.g. an example output file based on above (cutting out the first section) would be:

http://www.webfound.net/flatfile_output.txt

Also, I'll need to grab a certain value in each header row (certain position in the 100 byte header row) to use that as part of the filename that's outputed.  I assume it would be better to insert these rows into a temp table then somehow do a search on a specific position in the row...but that's impossible?  The other route is to insert each row into a temp table separated out by fields but that is going to be too combursome because we have several formats to determine separation of fields based on the row type so I'd have to create many temp tables and many components in SSIS when all we want to do is again:

1) output each group (broken by each header row) into it's own txt file

2) use a field in  the  header row as part of the name of the output txt file (e.g. look at the first row, whcih is a header row in flat_file_example. txt.  I want to grab the text 'AR10' and use that as part of the filename that I create

Any suggestions on how to approach this whole process in SSIS...the simplest approach that will work ?

View 1 Replies View Related

Flat File To Table

Jun 11, 2007

Hi,

I have a set of flat files and transforming it to SQL server. If I do that in 2000 it was done with in 45 seconds for 1.5 M records. If I do the same in SSIS it takes 3 minutes. Why there is difference in time that too lower when compared to the previous version. I used the data access mode as "Fast load". Am I missing anything while doing through SSIS?

View 11 Replies View Related

Flat File To SQL Table

Jun 11, 2007

I am looking to evaluate a text field in the row and change it to an integer in the sql table based on the text.



What is the best data flow tranformation for me to use inbetween the flat file source and the ole db destination?

View 7 Replies View Related

Pivot Table/Flat File - HELP!

May 16, 2008

Ok,I revamped my query, I thought I fa different way.....

Select
RecordID,
Type, (Traditional, Teclar, AllClaims)
Year,
Month,
TotalCharges,
AdjustedTotalSavings,
AverageClaim,
PercentSavings,
DHSIncome
From sumClaimsSettlement

What I need the end result to look like...

Type Q1 Q2 Q3 Q4

Traditional
Teclar
All Claims

Under the Quarter cloumn will be Total Charges. What is the better way to do this? Should I go with a Pivot table or make it into a flat file?

View 2 Replies View Related

How To Create Sql Table From Flat File

Jun 4, 2007

Any one can help for how to create sql table dynamically as per flat file format...
its required very urgently

View 1 Replies View Related

Unload A Table To A Flat Or Raw File

Mar 26, 2008

Is there any faster and efficient way for exporting or unloading the data from a table (Contains about 400 to 500 million rows) into a flat or raw file. I am using SQL server 2005 and the Export Utility in SQL Server is very slow.

Any help is greatly appreciated...

View 1 Replies View Related

How To Dump Records From The Table To The Flat File?

May 21, 2001

I'm using sqlserver 2000 enterprise edition. I am an oracle dba and we have some tables in sqlserver 2000 that we need to write out to the flat file. I have a procedure in oracle to do this for oracle tables. But, how would I do this in sqlserver 2000. I have 10 columns on this table and I only want 3 columns data to be dumped on the flat file. We are on NT sever 4.0.

Thanks,
Ranjan

View 1 Replies View Related

Reference A Flat File (as A Table) In A Sql Statemtent

Apr 5, 2004

Several years ago, I am sure that I had a query that either read from or wrote to a text file. There was some way to:
select * into 'c:\out.txt' from SomeTable
Does anyone know if this functionality exists? If so, what is the syntax?

TIA

View 8 Replies View Related

Pulling Data From Flat File To A Table

Nov 2, 2007

I need to do some advanced formatting in the text file, because the data in the text file is bit complex. I have some knowledge of using BCP, BULK INSERT and bit about FORMAT FILES but I want to do some advanced formatting(using IF condition and all) to pull the data from text file.
Can anyone please tell me how can I perform bit advanced formatting using BULK INSERT?

Cheers

View 10 Replies View Related

Using SQL To Append Records To A Table From A Flat File Using DTS

Aug 4, 2006

What I would LIKE to do is noted in the subject line. What I'm findingis that "edit SQL" appears to only be an option if I am creating atable. If I select "append to" the option to edit SQL shades itself asunavailable.The reason I'd like this is that there is a datum in the flat file thatindicates whether that record should be appended to that table notedabove. There are other ways of dealing with this "problem" but it wouldbe nice to be able to control it using SQL, in the DTS import/exportwizard.If the source of my data is an SQL table, I can generate an SQL queryto specify what fields to import in an append, to check for existingvalues, etc...Is there a way around this? I can reserve a table for data transfers,regularly overwrite it with new data from text file inputs, and use SQLto insert select fields from that transfer table to other databasetables. (From this "transfer" table, data needs to be inserted intofour separate tables in our database).I hope this is clear. If it CAN'T be done this way, it's okay...just alittle ugly with the need to re-create the transfer table.

View 3 Replies View Related

Update Or Upsert A Table From A Flat File

Jul 24, 2006

Hi All,

I have read what people have posted on this but don't seem to find exactly what i want. I tried the methods from this website like many suggest http://www.sqlis.com/default.aspx?311. But all this seems to do insert new records but does not update the existing records.

This is what i want to do. I have an online store that uses an SQL DB with a products table. I have an excel/csv spreadsheet with the same columns as in the sql database. It is much quicker for me to update the spreadsheet than logging in to sql to do it. So i would like to update all the product prices on flat file to the sql table. There will be more updating than adding new rows.

Please can you help.

Thanks

View 1 Replies View Related

Flat File To Table - Rows Out Of Order

Jun 12, 2007

Hi,



I noticed something strange today. I created a pkg that reads a flat file and writes the rows to a table.



In checking the data in the file against what's in the table, I noticed that the rows were inserted in a different order than they are in the file.



All the rows appear to be in the table correctly, but they're just not in the same order as in the file. I've never seen this before. But I checked very carefully, and this is indeed the case.



Is this normal??



Thanks

View 8 Replies View Related

Insert From Flat File Where Column 1 Not In Table

Aug 7, 2007

Hi Guys,

I hope this is easy stuff for you..
First of all i searched the forum but didnt exactly find what iam searching for:

I have a File folder which contains 1..n Files of the same type.
The files contain a DateValue at the beginning of each row. I now want to read the first Record of the file - extract the datevalue and search in my Importtable if there are any records with that Date. If there are allready Records with this date i know i allready imported that file and skip it in my For each File Container. If no records where found I want to copy them from the file to the table.

So I have a flatfile source and thoght I just make an oledb command task afterwards which looks like "Select count(*) from Import where Processdate = ?. and then a conditional split if the count == 0 or not... but i have problems getting the Count value out auf the OLD DB Command Task because everytime I try to add an outputcolumn i get the message: "An Output cannot be added to the output collection" and since there is no possibility to map an expression to the result...


I tried to workaround the problem using a lookup task.. but that seems to be the wrong way.

thanks for your help
bye
AS

View 3 Replies View Related

Row Count From Flat File To Table In The Target

Aug 22, 2006

We have project in SSIS:

Control Flow
Load Data Flow Task to Copy Data Flow Task to Scrip Task

Data Flow under
Load Data Flow Task has
Flat File Source to Row Count1 to OLE DB Destination (ODS database)
Variable name for Row Count1 is RowCount

Data Flow under
Copy Data Flow Task has
OLE DB Source to RowCount2 to OLE DB Destination (WIMS database)
Variable name for Row Count2 is RowNumber

Data Flow under Script Task is code:

' Microsoft SQL Server Integration Services Script Task
' Write scripts using Microsoft Visual Basic
' The ScriptMain class is the entry point of the Script Task.

Imports System
Imports System.Data
Imports System.Math
Imports Microsoft.VisualBasic
Imports System.Diagnostics
Imports Microsoft.SqlServer.Dts.Runtime
Imports System.IO



Public Class ScriptMain

' The execution engine calls this method when the task executes.
' To access the object model, use the Dts object. Connections, variables, events,
' and logging features are available as static members of the Dts class.
' Before returning from this method, set the value of Dts.TaskResult to indicate success or failure.
'
' To open Code and Text Editor Help, press F1.
' To open Object Browser, press Ctrl+Alt+J.

Public Sub Main()
Dim varMyRowCount As Variable = Dts.Variables("RowCount")
Dim varMyRowNumber As Variable = Dts.Variables("RowNumber")
Dim varPackageName As Variable = Dts.Variables("PackageName")
Dim varStartTime As Variable = Dts.Variables("StartTime")
Dim varInstanceID As Variable = Dts.Variables("ExecutionInstanceGUID")
Dim varMailMsgtext As Variable = Dts.Variables("MailMsgText")
Dim PackageDuration As Long
Dim Filenum As Integer
Dim FilNam As String



'<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
' Event log needs
'>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
Dim sSource As String
Dim sLog As String
Dim sEventMessage As String
Dim sMachine As String
'<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

PackageDuration = DateDiff("s", varStartTime.Value, Now())

sSource = "RowCountReporting from SSIS"

' We need the message posted to the Application event log.

sLog = "Application"
sEventMessage = "Flat File Row Process: " & Chr(10) _
& " case Rows:" + varMyRowCount.Value().ToString + Chr(10) _
& "Wims Rows Processed: " & Chr(10) _
& " case Rows:" + varMyRowNumber.Value().ToString + Chr(10) _
& "Started: " & varStartTime.Value().ToString _
& Chr(10) _
& "Current Time:" & System.DateTime.Now _
& Chr(10) _
& "=============================================" _
& Chr(10) _
& "Package Run Duration in seconds: " & PackageDuration _
& Chr(10) _
& "Execution GUID: " & varInstanceID.Value().ToString
sMachine = "."

varMailMsgtext.Value = sEventMessage

If Not EventLog.SourceExists(sSource, sMachine) Then
EventLog.CreateEventSource(sSource, sLog, sMachine)
End If

Dim ELog As New EventLog(sLog, sMachine, sSource)

ELog.WriteEntry(sEventMessage, EventLogEntryType.Information, 777, 2)


Dts.TaskResult = Dts.Results.Success
End Sub

End Class


We want to verify flat file count should be same as the data load to WIMS database.
Now if I run this job I get the error message:
Error: Failed to lock variable "RowCount" for read access with error 0xC0010001 "The variable cannot be found. This occurs when an attempt is made to retrieve a variable from the Variables collection on a container during execution of the package, and the variable is not there. The variable name may have changed or the variable is not being created.".

My co-work said even I get balance from RowCount1 and Rowcount2 still need to have query from table in Wims database because for SQL server sometimes process counts not same as the data results load in table so need select count(*) from table. How and where I can put at the job.

My design is row count from flat file put one variable then row count from final table put the other variable then send e-mail to me show both variable for testing.
For real job I need if both variables same ok else send me an e-mail.

Any one has any idea? I will very appreciate.


Thanks.
Grace

View 1 Replies View Related

Create A Flat File For Each Record In A Table

Apr 20, 2006

I'm rather new to ssis and I've been reading and testing but didn't find a solution for this problem.Supose I've got a table Customer with some fields. One of the fields is CustID.I want to create as many flat files as there are Customers in the table with flat file name set to the CustID.If you could point me in a good direction, It would be nice.Greetings from Belgium

View 4 Replies View Related

Update Table From Flat File Source

Aug 31, 2006



Hi all,

I'm trying to work out how to update a table from a flat file source.

I could use the simple way of using OLE DB Command with

UPDATE <tblName> SET Column1 = ?, Column2 = ?, <etc> WHERE ID = ?

then using the mapping to join the parameter fields.

However, this (maybe!) a slow process when you have potential 500k+ records in the source file (and approx 150 files to process!)

What I would like to do is the following statement but I'm not sure if this is possible?

UPDATE <tblName> SET Column1 = sourceColumn1, Column2 = sourceColumn1, <etc> FROM <sourcefile> WHERE ID = sourceID

I've also thought about importing the sourcefile into a Recordset Destination and trying to use the variable in the SQLCommand but I'm not sure how to do this...

Does anybody have any ideas on how this could work or an alternative solution?

Thanks in advance....

View 4 Replies View Related

How To Redirect The Flat File's Data Into Another Table ?

Aug 28, 2007


If the system not able to convert the data from one data type(string,int) to another data type then i need to move the reocord into my error_table.

How to redirect the bad data into another table, while extracting the flat file data ?

Bad Data means if the system not able to convert the data from one data type to another data type.

Say X2R322? data cant convert as DATE data type but "070899" can be converted as DATE data type.

Thank you very much.

View 6 Replies View Related

Bulk Insert - Flat File Data Into Table

Mar 12, 2015

I am running a set of SQL statements on a SQL server, to insert flat file data into a SQL table. The flat file is already FTP'ed to the SQL server. I seem to be getting an error, which is possibly pointing to a permissions issue

The statements:

BULK INSERT [Jedox_prod].[dbo].[B_BP_Customer]
FROM 'c:jedox_dailyjdcom4401.txt'
WITH
(
FIRSTROW = 2,
MAXERRORS = 0,
FIELDTERMINATOR = '|',
ROWTERMINATOR = '
'
)
GO

The error is :
Msg 4861, Level 16, State 1, Line 1
Cannot bulk load because the file "c:jedox_dailyjdcom4401.txt" could not be opened. Operating system error code 3(failed to retrieve text for this error. Reason: 1815)

If it is permissions issue, how do I overcome this?

View 1 Replies View Related

Flat File Data Types - Defined From Table?

Jul 3, 2007

This question is around how we can get the data types and lengths populated into the flat file source columns.



In Connection Manager, you have your flat file defined. You can choose "Suggest Types...", and the minimum lengths and correct data types will be returned from within the data in the flat file.



Is there some way to automate this data type definition, but coming from the other direction (coming from the destination table that we are loading)?



For example, you have mapped the columns that will be loaded. Can you then reverse engineer the data types and lengths for the columns in the flat file from the destination table?



TIA

View 5 Replies View Related

Transform Header And Detail Flat File Into One Table

Sep 13, 2007

I have a flat file with header and detail information, it is actually employee punch card data. I need to parse the header line which contains the Employee ID and don't save it to a table just save the value. Then with the detail line, parse the different data elements and save them along with the employee ID to one table. Then continue until the next header line is read.


The file looks something like this:

FINNEY,RONNIE 0001005420
Mon 09/03 700a HOL 8.00
Tue 09/04 630a*E 326p 8.50 8.50
Wed 09/05 645a 330p 8.00 16.50
Thu 09/06 639a 2.40 18.90
HALL,MARK 0001005601
Mon 09/03 700a HOL 8.00
Tue 09/04 608a*E 257p 8.40 8.40
Wed 09/05 601a*E 259p 8.50 16.90
Thu 09/06 606a*E 3.30 20.20
JONES,WILLA JEAN 0001005702
Mon 09/03 700a HOL 8.00
Tue 09/04 556a*E 326p 9.10 9.10
Wed 09/05 600a*E 328p 9.00 18.10
Thu 09/06 554a*E 3.50 21.60

So I think I need a data flow transformation object that let's me save the Employee ID into a variable available when the next record is read. What type of transformation would be best?

View 1 Replies View Related

How Can I Output Subset Of A Table To A Flat File In SSIS?

Aug 24, 2006

Hi All,

I have a table A. I need output subset of a table A to a flat file using query, like:

select A.* from A inner join B on... ..... inner join C......where left(A.id, 3) = B.sid.... AND B.num between 100 and 200).

How can I do this in SSIS? Which data flow item I may need?

Thanks

 

View 2 Replies View Related

Update Server Table Based On Flat File Data

Feb 13, 2014

I have a table dbo.Sales that contains all sales records. There is a column in that table called ItemNumber that I'd like to match with ItemNumber in a flat file and update the ItemCost based on the ItemCost column in the flat file.

So while there will be many sales records for each ItemNumber, I need to loop through and update the ItemCost in that sales record based on the corresponding ItemCost in the flat file. Does this make sense? I really need this for court and I can't figure out how to do it. I took a SQL course about 7 years ago but have forgotten everything.

Database Name: BTData
Database Table: dbo.Sales
Database Columns: ItemNumber (match on this), ItemCost (update this)

FlatFile Name: InventoryCosts.txt
FlatFile Columns: ItemNumber, ItemCost

There will be many sales records for each ItemNumber in the database table. I need to update each one with correct cost based on the item number and cost mapping from flat file.

View 1 Replies View Related

How To Validate Flat File Data Against Not Null Columns Of A Table?

Nov 27, 2007

Hi all,

I am facing a problem on validating the data from a flat file while inserting the data into the destination table of sql server 2005 database. In my package, i have to validate the input data whether the values are coming as null or not, before inserting into the destination database. The flat file may not contain data for all NOT NULL columns. I have to find out that row(s) and reject the record. If the rows are coming as Null for the Not Null columns, the OLEDB Destination throws OLEDB exception for the constraint.

To resolve this, i have an script component in data flow, to check whether the input data is coming as null. I have added the output column of boolean type to the script component, it will be assigned to TRUE when there is null for the Not Null column in the script.

And in the follwing conditional split, i am checking the flag for TRUE to reject the record.

Is there any other way to handle this validation?

Regards
Madhav

View 6 Replies View Related

Records From Internal Table Not Output In Order To Flat File

Oct 13, 2006

Hi -

I have a SSIS package that dumps data from an internal table to a flat file output using standard data flow tasks. The entire table is output - no special SQL. Most of the time the records are placed in the output file in the same order as the internal DB table, but occasionally the order appears to be more random. When that happens, the record order in the internal table is correct - it is just the output.

I can find no properties that seem to affect this. I would appreciate any hints and advice that anyone can give me. Has anyone else encountered this same problem?

Thanks in advance,

Jim Sweeney

View 4 Replies View Related

SSIS - Output Table To Flat Fixed-width File

Oct 19, 2007

I am new to SSIS and am having trouble with automatically setting up the destination output columns.

I am sure there must be an easy way to do this.

My table (source) has 86 columns in it of varying lenghts.

In my connection magagers, I have created one for the SQL Server (source data) and one for the flat file (destination output).

I have also created an OLE DB source data object and a destination Flat File object and set them up to the respective connection managers.

Finally I have linked the source to the destination.

Now when I look at the source, it shows me all 86 columns.

When I open up destination, there are no columns set up.

Problem: do I have to type in all the columns manually in the connection manager for the Flat File?

I would think there would be some automatic way that it would self-populate the columns over to the flat file destination.

View 3 Replies View Related

Integration Services :: Flat File Error File Being Created In-spite Of No Errors

Jun 23, 2015

I have a package in which there are only one Data flow Task and it has only three components. 1) Source , which is a SQL db 2) destination and 3) OLE DB Destination flat file Error output file.   I want the error file to be created ONLY if there is any error while dumping the data into destination DB. But , the issue is, the error flat file is being created inspite of No error while dumping the  data from Source to Destination.

View 5 Replies View Related

Integration Services :: Get FileName Fo Each File Created Via Dynamic Flat File Destination

Jul 24, 2015

Need to know how I can get the dynamic filename created in the FlatFile destination for insert into a package audit table?

Scenario: Have created a package that successfully outputs Dynamiclly named flat files { Format: C:Test’Comms_File_’ + ‘User::FileNumber’+’_’+Date +’.txt’

E.g.: Comms_File_1_20150724.txt, Comms_File_2_20150724.txt  etc} using Foreach Loop Container  :

* Enumerator Set to: “Foreach ADO Enumerator” with the ADO object source variable selected to identify how many total loop iterations there are i.e. Let’s say 4 thus 4 files to be created

*Variable Mappings : added the User::FileNumber – indicates which file number current loop iteration is i.e. 1,2,3,4

For the DataFlow task have a OLDBSource and a FlatFile Destination where Flat File ConnectionString is set up as:

@[User::Output_Path] + "Comms_File"+ @[User:: FileNumber] +"_" + replace((DT_WSTR, 10) (DT_DBDATE) GETDATE(),"-","")+ ".txt"

All this successfully creates these 4 files:

Comms_File_1_20150724.txt, Comms_File_2_20150724.txt, Comms_File_3_20150724.txt, Comms_File_4_20150724.txt

Now the QUESTION is how do I get these filenames as I need to insert them into a DB Audittable. The audit table looks like this:

CREATE TABLE dbo.MMMAudit
  (
     AuditID      INT IDENTITY(1, 1) NOT NULL,
     PackageName     VARCHAR(100) NULL,
  
FileName           VARCHAR(100) NULL,
     LoadTime        DATETIME NULL,
     NumberofRecords INT NULL
  ) 

To save the Filename & how many records in each file in our Audit Table, am using an Execute SQL Task and configuring it as this:

Execute SQL Task

Parameter mapping - Mapped the User Variable (RecordsInserted) and System Variable( PackageName) to Insert statement as shown below

SQLStatement: INSERT INTO [dbo].[MMMAudit] ( 
PackageName,NumerofRecords,LoadTime)
 (?,?.GETDATE)

Again this all works terrific & populates the dbo.MMMAudit table as shown below BUT I also need to insert the respsctive file name – How do I do that?

AuditID PackageName FileName  NumberOfRecords
1           MMM       NULL                      12
2          MMM  NULL                23
3          MMM  NULL      14
4          MMM  NULL              1                     

View 2 Replies View Related

SQL 2012 :: Find Out Number Of Columns In Flat File Before Process That Particular File

Apr 14, 2014

I need find out the number of columns in flat file before i process that particular file.I have file name in @filename variable and file path is @filepath variable.But do not not that how i will check the column name in before i will process that file.

@filePath = C:DatabaseSourceFilesCAHCVSSourceFiles
And i am using for each loop container to read the file one by one and put the file name in @filename variable.and my file name like

Product_20120607060930.txt
Product_20130708060930.txt

[code]....

Now what i have to do is i need to make sure that ID,Name,City,County,Phone is there in flat file.if it is not there then i have to send mail to client saying that file is not valid.I need to also calculate the size of flat file.

View 4 Replies View Related







Copyrights 2005-15 www.BigResource.com, All rights reserved