BCP Task And Dynamic Import And Export Of A File Via Package Variables

Jul 3, 2006

I have a requirement to create many SSIS packages and no datatransform is required so the BCP task looks a good contender providing it can do both import & export

is it possible to parse the values in bold as package variables into the BCP task. If so how?

BULK INSERT ipcs_wvg.dbo.extract
FROM 'D:IPCSextract.csv'
WITH (FORMATFILE = 'D:ipcsqueryextract.xml');

Thanks in advance

Dave

I have a global database called ETL Configuration for all my SSIS packages that uses a single table. So I can create three global variables

USE [ETLConfiguration]
CREATE TABLE [dbo].[SSIS Configurations](
[ConfigurationFilter] [nvarchar](255) COLLATE Latin1_General_CI_AS NOT NULL,
[ConfiguredValue] [nvarchar](255) COLLATE Latin1_General_CI_AS NULL,
[PackagePath] [nvarchar](255) COLLATE Latin1_General_CI_AS NOT NULL,
[ConfiguredValueType] [nvarchar](20) COLLATE Latin1_General_CI_AS NOT NULL
) ON [PRIMARY]

Here is the data I would put in here

ConfigurationFilter = 'MySSISPackageName'

ConfiguredValue = 'D:IPCSextract.csv'

PackagePath = 'Package.Variables[User::gsFileName].Properties[Value]'

ConfiguredValueType = 'String'



ConfigurationFilter = 'MySSISPackageName'

ConfiguredValue = 'D:ipcsqueryextract.xml'

PackagePath = 'Package.Variables[User::gsFormatFile].Properties[Value]'

ConfiguredValueType = 'String'

ConfigurationFilter = 'MySSISPackageName'

ConfiguredValue = 'ipcs_wvg.dbo.extract'

PackagePath = 'Package.Variables[User::gsTableName].Properties[Value]'

ConfiguredValueType = 'String'



--Database connection info

ConfigurationFilter = 'MySSISPackageName'

ConfiguredValue = '.mssql2005'

PackagePath = 'Package.Connections[MyDatabaseName].Properties[ServerName]'

ConfiguredValueType = 'String'



ConfigurationFilter = 'MySSISPackageName'

ConfiguredValue = 'MyDatabaseName'

PackagePath = 'Package.Connections[MyDatabaseName].Properties[InitialCatalog]'

ConfiguredValueType = 'String'



I have looked at lots of options to automaticly create SSIS packages and have a hunch that that simple can be better: All these solutions look way to complex to what I want to achieve--

http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=17484&SiteID=1

http://lakshmik.blogspot.com/2005/05/how-to-programmatically-create-ssis.html

Create DataFlow Package Sample
http://msdn2.microsoft.com/en-us/library/ms161541.aspx

SMOTableList Sample
http://msdn2.microsoft.com/en-us/library/ms161564.aspx

View 4 Replies


ADVERTISEMENT

Execute DTS 2000 Package Task Editor (Inner Variables Vs Outer Variables)

Sep 4, 2006

Hi,

I am not comfortable with DTS 2000 but I need to execute a encapsulated DTS 2000 package from a SSIS package. The real problem is when I need to pass SSIS variables to DTS 2000 package. The DTS 2000 package have 3 global variables that I can identify on " Execute DTS 2000 Package Task Editor - Inner Variables ". I believe the SSIS variables must be mapped on " Execute DTS 2000 Package Task Editor - OuterVariables ". How can I associate the SSIS variables(OuterVariables ) to "Inner Variables"? How can I do it? Much Thanks.

João





View 8 Replies View Related

How To Access Package Variables In A Custom Task

Jul 3, 2006

Hi Everyone,

I created a script task that uploads a file to a HTTPs site. The script task works well and now I want to turn this script task into a custom component...

I'm having problems on how to call the variables I have in the package, I already locked them for reading, but how am I suppose to access them and how can I put their values into strings so I can easily call them in the WebClient.UploadFile method?

Also, I am very new to custom tasks in SSIS.. and would appreciate if anyone of you can send me something (code, link, etc.) of a very simple custom task without any form or properties, that I can try to study.

Basically, what I want my custom task to do is,

1. Read the varialbes in the package.
2. use the variables values as input in my WebClient.UploadFile("varURLstring", "PUT", "varFilePath")

as you can see, my custom task is really simple... but I'm a noob here and I don't know where to go.

Thanks in advance,
Kervy

View 1 Replies View Related

Using Package Variables As Parameters In Execute Sql Task

Feb 23, 2006

Hi,

I need to run an SQL command, in which one of the fields is based on the current time, represented as a string in a specific format.

I used a script task to create the string from the current time, and store it in a package variable named "newDate". Later, in the execute sql task, I have the following update statement: "update table_1 set field_1 = ?" (OLE DB connection is used)

and in the parameters mapping I set parameter 0 to the string package variable "newDate".

The problem is that on runtime, the sql executes with the default value of the variable - i.e. it doesn't take the new value.

I checked it and placed another script task after the first one, that simply tries to read the variable "newDate" and print it to screen, but even here the default value is used.

how can I use the new value of the variable?

thanks.

View 7 Replies View Related

Package Variables Passed To Execute SQL Task

Aug 14, 2007

Hi,

I am having trouble getting the Execute SQL Task to recognize the package-level variables in an SSIS package. The tasks fail execution. The package contains three Execute SQL tasks and none of them receive the variables. However, when I replace the '?' in the code with the variable values they execute fine. These are the steps I have taken thus far:

Made sure the three variables are package level.
Verified I'm using an OLE DB valid connection.
Verified the variables are properly mapped in the parameters mapping section of the task properties.
Tried all 4 types of variable to parameter mapping (even though '?' in the query and '0,1,2..' in the name should work).
Changed the Bypassprepare property to 'True'.
Breakpoints and watch windows to verify the variable values.



I actually tried the simple tutorials on a one-liner SQL statement that uses variables, but had the same problem. I know there must be something simple that I'm missing here, but any info would be greatly appreciated!

Thanks!

View 5 Replies View Related

How To Run Import / Export Package On Computer

May 8, 2014

I created a package from the SQL Server Import and Export Wizard.I can run this .dtsx file " Import_Table.dtsx " on my local machine no problem, which then executes the package on the remote server of my hosting company.Do they need to install SQL Server on their machine for it to work?Is there some files they can download to make it work?

View 3 Replies View Related

Editing An Import/Export Package

Apr 17, 2008

Hi,


First post, new to SQL Server, the usual apologies apply ;-)


I used the Import/Export Wizard in SSIS to define a source and target for data migration. I accidentally omitted ticking off a box to force the drop and re-creation of the target table. I can't find any way to edit a package, once I've finished with the wizard! If someone would point me in the right direction, I'd appreciate it.


Thanks,
Chap

View 4 Replies View Related

Export Or Import A Dataflow To Another Package

Mar 28, 2008

Hi, I have one package with a complex Control Flow. I need to import a dataflow from another package into it. Cut and Paste is working but it looks like my package corrupts after that. (for example, login to database has problem with authentication now)
Is there any import or export tool for this?
Regards

View 5 Replies View Related

Passing Global Variables From A Execute Package Task

Apr 28, 2004

I have a package (Package1) that is run from another package (Package2) via a Execute Package Task. I set a Global Variable called sErrorMessage in the in Package1 and would like to access that Global Variable in an ActiveX Script Task in Package2. How can I do this?

View 6 Replies View Related

Execute Package Task: Setting Child Variables

Feb 13, 2007

Hi,
Let's say I have a package taking as parameter "InvoiceID". I want to execute this package as a child in another package. The parent package gets the list of invoices to produce and calls the child package for each entry of the list.

How do I pass the InvoiceID to the child? I know I can use the parent's variables from the child but I don't want the child package to be dependant on the parent package. For example, I might want to execute the "child" package as a stand-alone in development (providing it with a predefined InvoiceID). I might also want to call the same child package from another parent package with other variable names.

What I would like to do is "push" the value instead of "pulling" it. I know it's possible using the command line and the /SET option (ex.: /SET Package.Variables[InvoiceID].Value;' 184084)... Is it possible using the Execute Package Task?

Thanks

View 4 Replies View Related

How To Read The Names Of Package Variables In Custom Task

Nov 22, 2005

Hi there,

View 4 Replies View Related

Custom Task: How To Access/modify The Variables Of A Package

Feb 1, 2007

Hi,

I'm trying to simplify the deployment process of my project. I already had some troubles with the config files but lets say I solved that issue. I'm going to read a flat file and set the variables of my packages from this file. I was thinking to use a Script Task to do that but I will need to copy this task in every package (I have at least 30). So if I want to make some change this will be painful.

Then, I came up with the idea of creating a Custom Task called Config File Task. I'm working on this but I got stuck trying to get the variables from the package that is running my Config File Task.

This is the code I had in the Script Task:

Dim streamReader As New StreamReader(Dts.Variables("ConfigFilePath").Value.ToString)
Dim line As String
Dim lineArray As String()
Dim variableName As String
Dim variableValue As String
Dim readConfigurations As Boolean = False

While (streamReader.Peek() <> -1)
line = streamReader.ReadLine()
If line = "[CONFIGURATIONS]" Then
readConfigurations = True
ElseIf line = "[/CONFIGURATIONS]" Then
readConfigurations = False
Else
If readConfigurations And line <> "" Then
lineArray = line.Split("|".ToCharArray())
variableName = lineArray(0).Trim()
variableValue = lineArray(1).Trim()

If Dts.Variables.Contains(variableName) Then
Dts.Variables(variableName).Value = variableValue
End If
End If
End If
End While
Dts.TaskResult = Dts.Results.Success

All I want to do is set the variables that exists in each package from the config file. In my UI Class (ConfigFileTaskUI.cs) I can have access to the variables via the TaskHost which is passed as an argument of Initialize() method.

Any thoughts? I'd really appreciate some help!

P.S. I've been working on this for 2 days!

View 11 Replies View Related

Where The Saved Import Export Package Listed

May 29, 2007

Hi,
My Warm Wishes to All !!

I ve created a Import Export Package by right clicking the specified Database and Save it as SQL Server Type thru the wizard and execute immediately , and it works . But I dont know where it get saved and cant able to see so that I could be able run once more. More over I want to schedule the package for that what I have to do ??? Plz help me

Thanks

Aravind

View 2 Replies View Related

Running An SSIS Package After Using Import/Export Wizard

Dec 17, 2007

Hello All,

I am a newbe at MS SQL 2005, so if this has already been answered elsewhere, please just point me in the right direction.

I have successfully used the Import/Export wizard in "execute immediate" mode to import a table from Oracle 10g to MS SQL 2005. Works like a charm, but when I take the saved pacakge (saved as a *.dtsx server side file) - with no changes to the package and execute it from the either the "dos" prompt or from Windows explorer it fails at the login to Oracle step. Yet when I check the package's "Connection Managers" source connection, all the settings, user-ids and passwords look fine. The package does indeed execute, but I receive an ORA-01017:invalid username/password within the trace file of the package, but nothing has been changed from the Import/Export wizard which ran beatifully, thus verifying the appropriate acccess rights to Oracle.

Any advice would be greatly appreciated.

Doug

View 7 Replies View Related

HowTo: Read/write Package Level Variables In Custom Task

Oct 13, 2006

Hello all,

I have been struggling trying to read and/or write package level variables from within my custom task.  I'd like to be able to get and set values from within the Execute method of my custom task.  I have searched this forum and the books online and can't seem to find the answer.  I thought maybe I could use an expression on my task (mapping the package variable to a custom task public property) but that doesn't seem to be working for me.  I also would have thought I could use the VariableDispenser object from within my task but the collection is empty.  I have 3 package level variables configured and can't seem to find a way to access them (with intentions of getting/setting).  Could someone point me to a good doc or provide an example that may accomplish this?  Thanks!

(I'm using package level variables as a means of passing simple information between tasks that are not using a DB, if there is a better way I'm open to suggestions.)

Jay_G

View 3 Replies View Related

Execute Package Task - Dynamic Name

Nov 7, 2007

Hi,

Is there anyway where i can pass the package name to an "execute package task" and run it through loop. My loop task will pass the package name which is assigned on a variable and the execute package task should pickup the name and run, is this possible, if yes how?

View 3 Replies View Related

Using Data Source View In Import Export Package Within Same Solution Is Not Possible ?

Dec 15, 2005

hello ,

 

I want to download data from an AS/400 to a SQL 2005 Database.

Within Sql server 2000 it was possible to do more than 1 transformation with just 1 source connenction and 1 destination.

In SSIS I have created:

-  new solution

-  data source

-  Data source viewer ( I made named queries to format the data as i want it to be in Sql)

Although it is possible to view the named query data it does not seem to be possible to import the data in the data viewer into my sql 2005 database. (For each dataflow I have to define each query again)

I have added the data source in the connection manager. 

Also I am searching to import more than 1 file in 1 package !

I can use however a Dts-2000 package but why then use SQL server 2005

Hope somebody can help me

Regard,

Ronny

 

View 7 Replies View Related

File Task Moving With Variables...

Apr 25, 2007

I have a 'file system task' moving files from one server to another for processing. I have defined the path and filename as separate variables. When I attempt to pass them together into the task I receive an error.



Source Path & Filename:

@[User:: DataSourceFolder]+"\"+ @[User::CD_PaidClaimSource]

Outputs:

\umrdwh2FTPCLAIMDAILY.DATA



Destination Path Now: (Filename not specified per another thread.)

@[User::WorkingFolder]



Original Destination Path: (Which did not work.)

@[User::WorkingFolder]+"\"+ @[User::CD_PaidClaimSource]



I receive validation errors:

Error 1 Validation error. CLAIMDAILY Move : Failed to lock variable "\umrdwh2FTPCLAIMDAILY.DATA" for read access with error 0xC0010001 "The variable cannot be found. This occurs when an attempt is made to retrieve a variable from the Variables collection on a container during execution of the package, and the variable is not there. The variable name may have changed or the variable is not being created.". ClaimLoading_MASTER.dtsx 0 0





How do I overcome this without hard coding the path in the task?












View 11 Replies View Related

File System Task - Move File With Dynamic Destination Path

Oct 9, 2006

I am having an issue with the File System Task.

I was wondering if there is a way to 'Move File' with the File System Task inside of a For Each Loop container but to dynamically set the Destination path variable.

Currently, this is what I have:
FileDestinationPath variable - set to C:TestFiles
FileSourcePath variable - set to C:TestFiles
FileNameAndLocation variable - set to blank

For Each Loop Container €“ Iterates through a folder C:TestFiles that has .txt files in it with dates in the file name. Ex: Test_09142006.txt. Sets the file path (fully qualified) to the Variable Mapping FileNameAndLocation.

Script Task (within For Each Loop, first step) €“ Sets the FileDestinationPath to the correct dated folder within C:TestFiles. For example, if the text files I want to move are for the 14th of September, it takes FileDestinationPath and appends the date folder to the end of it. The text files have a date in the file name (test_09142006.txt) and I am picking this apart (from FileNameAndLocation in the For Each Loop) to get the folder date. (dts.Variables(€œUser::FileDestinationPath€?).Value = dts.Variables(€œUser::FileDestinationPath€?).Value & €œ€? Month & €œ_€? & Day & €œ_€? & Year & €œ€?) which gives me €œC:TestFiles9_14_2006€?.

File System Task (within For Each Loop, second step) €“ This is where the action is supposed to occur. I want it to take the FileDestinationPath and move the FileNameAndLocation file (from the For Loop) into this folder for each run.


Now as for my problem. I want this package to run everyday but it has to set the FileDestinationPath variable dynamically according to that day€™s date. Basically, how do I get this to work since I can€™t hard code the destination path variable from the start? I have the DestinationVariable on the File System Task set to the FileDestinationPath variable, after the script task builds it. However, using FileNameAndLocation as the SourceVariable on my File System Task tells me that the €œVariable €œFileNameAndLocation€? is used as a source or destination and is empty.€?

Let me know if I need to clarify further€¦...I may be missing something very simple. Any help would be greatly appreciated!

View 10 Replies View Related

How To Import And Export CSV File

Aug 25, 2007

Hi frnz ,
i want to know how to import a CSV File and export CSV File From database SQLSERVER-2005 Using ASP.NET With C#
 
IF any one knows Help me
Vinnu 
 

View 1 Replies View Related

SQL 2012 :: Passing Record Counts From Execute Task In SSIS To Package Variables

Mar 4, 2014

I've got a package in SSIS 2012 that has an Execute SQL task in the control flow level.

The SQL in question does an Upsert via the SQL merge statement. What I want to do, is return the count of records inserted and records updated (No deletes going on here to worry about). I'm using the output option to output the changed recs to a table variable.

I've tried returning the values as:

Select Count(*) as UpdateCount from @mergeOutput where Action = 'Update'
and
Select Count(*) as InsertCount from @mergeOutput where Action = 'Insert'

I've tried setting the resultset to both Single rowset and Full rowset, but i'm not seeing anything returned to the package variables I've set for them (intInsertcount and intUpdatecount).

View 2 Replies View Related

Import/Export From .csv File Into A Datatable

May 25, 2006

I'm using a DTS package to copy data from a comma delimited text file into a datatable. The text file has close to 160,000 records. The transfer fails at record 120,000. The error message says "Column Delimiter not found". The particular row in the .csv file seems to have nothing wrong with it as far closed quotes and missing commas are concerned. Even if I delete the row, the error is thrown at the same row.
Microsoft has acknowledged this as a bug in SQL Server 2000 and has a service pack to fix it. But installing SP4 is not an option for me at this time. The workaround on Microsoft websire about adding a character or a line didn't work for me.
Is it possible that there's some character or a marker in the text file that stays even if I delete the row? I don't know how to fix this problem. Does anyone here knows a workaround?
 

View 7 Replies View Related

File System Task - Dynamic Source File Name

Feb 27, 2008



Hi All,

I have a source files folder where the files generated everyday.
My goal is pick the latest file and copy this single file to another folder.
I used the Foreach loop container and got the latest file and stored the file name to a varible i.e. LatestFile
Then i want to use the File System Task to copy this to the destination.
On the beginning, I could not setup the Latestfile since I don't its name then, so when I setup the Source Connection property of the File system task, it is not allowed to leave the SourceVarible as blank!

Any suggestion?


Thanks

Micor

View 3 Replies View Related

Integration Services :: Package Location When Running Data Import / Export Wizard

Apr 23, 2015

Where is a package visible when running the Data Import/Export wizard, choosing to save a package, and choosing "SQL Server" as the location? When I make an SSIS connection in Management Studio I do not see the package under the "MSDB" node.

View 4 Replies View Related

Import A Dynamic File Name Into A Known Table

Sep 26, 2006

How would I import a dynamic file name into a known table?I have the file name in a variable. So what object in SSIS do I use to do this. Thanks

View 2 Replies View Related

Import/Export CSV File Via Stored Procedure

Jun 12, 2008

Reading through the forums, I found some great imformation for importing/exporting an excel spreadsheet via a stored procedure, however, the amount of data I have won't fit in excel. Can someone help me Import a CSV file via a stored procedure? and Export a CSV file via a stored procedure? I don't know if XML is the answer or if there is another way. For many reasons, I don't want to use DTS or Bulk. We are currently using SQL Server 2000.

Thank you,
Stacy



Thanks,
Stacy

View 1 Replies View Related

SSIS Import/Export Flat File

Oct 11, 2006

Firstly, I hope this question isn't asked too frequently but I found no existing reference to this situation....

I had a bunch of stored procedures in SQL 2k which imported and exported data to and from flat files using TEXTPTR, READTEXT, UPDATETEXT etc... The flat files were continuously changing so the filepath was a parameter for the sp.

The reason I used the pointer to flat files is because I didn't want to
load the files in memory before commiting them ie. with TEXTPTR and
UPDATETEXT I can import a 1Gb binary file 80000 bytes at a time and
keep (precious) memory usage down.


I was accessing this procs from a C# application.

Since these methods are going to be phased out by the guys at MS what is the best way of importing/exporting very large binary files in SQL 2005?

As far as I can tell SSIS requires a Flat File Source Manager object which needs a static filepath - not good.

Hope you can help,
Paul

View 10 Replies View Related

Export To Flat File Using T-SQL And Import To Another Machine

Jan 9, 2008



Hi,

I need to export some data from SQL 2005 to a flat file, The data and flat file names will be dynamic and will be be fired programaticaly so I can't use DTS or SSIS.

In SQL2000 I did it using bcp, but that's quite a security hole so I don't want to use external utilities. I'll need to do something similar on another machine to import the data as well.

I find it strange there's no easy way to do this!

Thanks.

View 4 Replies View Related

How Can I Export/import Data On Table To/from Text File

Dec 24, 1998

I needs export data on table to text file so I can process this data
with another database engine ie. Informix.

Can anybody help me to solve this problem ?

View 2 Replies View Related

Import Excel File Into Dropdownlist Then Export To Sql Server 2005

Jun 13, 2006

i am handling a project where user can choose the excel file and the field in the excel file to export into sql server 2005. which mean there will be dropdownlist where the user can choose the field and so on. anyone know how to do this?

View 2 Replies View Related

Importing Text File Problem Using Import And Export Wizard

Feb 5, 2007

Hi,

I'm trying to import text file (generated by UNIX - collation ISO LATIN 2) into the database using SQL SERVER 2005 Import and Export Wizard. I have got a problem with importing a decimal number, because in that column are not only decimal numbers (that's OK), but there are also spaces (not null, the column is filled by spaces and it looks like | |). When I'm trying import that file, then will occur the problem of truncation and import stops.

I can import that data by BULK INSERT, but I would like to import it by Import and Export Wizard at once without using subsequent conversions.

View 2 Replies View Related

Header Rows To Skip On Flat File Import/Export

Aug 29, 2006

Just attempting to import a simple tab delimited text file into my SQL Server 2005 database using the SQL Server Import and Export wizard. Column names are specified within the first line of the file. The Header Rows to Skip field value is listed as 0, but the wizard indicates that "The field, Header rows to skip, does not contain a valid numeric value".

Why isn't zero (0) a valid numeric value? I don't want to skip any rows. PLUS, I get the same error when trying to export to a text file although the header rows to skip field does not exist. I can increase the number to 1 or more, but the wizard will skip part of my data .. unacceptable.

What am I missing here? I installed SP1 of SQL server 2005, but that did not help.

Thanks in advance.



View 1 Replies View Related

Export File Using The DTS Package

Feb 20, 2008

Hi,

I have a DTS package scheduled to run every day creates a text file on Server A and then moves the text file from Server A to Server B. But recently, the package is moving incomplete text file from Server A to Server B. Not every day. There is no problem in generating the text file.

So far, I verified the memory on Server B - No Probelms.
Changed the package so it copies the text file from Server A to Sever B instead of moving, then delete the text file on Server B. - Didn't work out.

Any idea?

View 3 Replies View Related







Copyrights 2005-15 www.BigResource.com, All rights reserved