I am working on modifying a VB6 app that dynamically creates DTS packages to copy data from one database to another depending on the selections made in UI. The project currently uses DTSPackage object library and DTSDataDump Scripting object library.
We are in the process of upgrading the server to SQL 2005. I am exploring the possibility of replacing code that generates DTS packages on the fly with SSIS packages.
Is it feasible to do this in VB6 ? I have referred to similar posts which focus mainly on VB.NET or C#.
Any help with white paper or sample code would be appreciated.
I was intended to write a program that will create a SSIS package which will import data from a CSV file to the SQL server 2005. But I did not find any good example for this into the internet. I found some example which exports data from SQL server 2005 to CSV files. And following those examples I have tried to write my own. But I am facing some problem with that. What I am doing here is creating two connection manager objects, one for Flat file and another for OLEDB. And create a data flow task that has two data flow component, one for reading source and another for writing to destination. While debugging I can see that after invoking the ReinitializedMetaData() for the flat file source data flow component, there is not output column found. Why it is not fetching the output columns from the CSV file? And after that when it invokes the ReinitializedMetaData() for the destination data flow component it simply throws exception. Can any body help me to get around this problem? Even can anyone give me any link where I can find some useful article to accomplish this goal? I am giving my code here too. I will appreciate any kind of suggestion on this.
Code snippet:
public void CreatePackage() { string executeSqlTask = typeof(ExecuteSQLTask).AssemblyQualifiedName; Package pkg = new Package(); pkg.PackageType = DTSPackageType.DTSDesigner90; ConnectionManager oledbConnectionManager = CreateOLEDBConnection(pkg); ConnectionManager flatfileConnectionManager = CreateFileConnection(pkg); // creating the SQL Task for table creation Executable sqlTaskExecutable = pkg.Executables.Add(executeSqlTask); ExecuteSQLTask execSqlTask = (sqlTaskExecutable as Microsoft.SqlServer.Dts.Runtime.TaskHost).InnerObject as ExecuteSQLTask; execSqlTask.Connection = oledbConnectionManager.Name; execSqlTask.SqlStatementSource = "CREATE TABLE [MYDATABASE].[dbo].[MYTABLE] ([NAME] NVARCHAR(50),[AGE] NVARCHAR(50),[GENDER] NVARCHAR(50)) GO"; // creating the Data flow task Executable dataFlowExecutable = pkg.Executables.Add("DTS.Pipeline.1"); TaskHost pipeLineTaskHost = (TaskHost)dataFlowExecutable; MainPipe dataFlowTask = (MainPipe)pipeLineTaskHost.InnerObject; // Put a precedence constraint between the tasks. PrecedenceConstraint pcTasks = pkg.PrecedenceConstraints.Add(sqlTaskExecutable, dataFlowExecutable); pcTasks.Value = DTSExecResult.Success; pcTasks.EvalOp = DTSPrecedenceEvalOp.Constraint; // Now adding the data flow components IDTSComponentMetaData90 sourceDataFlowComponent = dataFlowTask.ComponentMetaDataCollection.New(); sourceDataFlowComponent.Name = "Source Data from Flat file"; // Here is the component class id for flat file source data sourceDataFlowComponent.ComponentClassID = "{90C7770B-DE7C-435E-880E-E718C92C0573}"; CManagedComponentWrapper managedInstance = sourceDataFlowComponent.Instantiate(); managedInstance.ProvideComponentProperties(); sourceDataFlowComponent. RuntimeConnectionCollection[0].ConnectionManagerID = flatfileConnectionManager.ID; sourceDataFlowComponent. RuntimeConnectionCollection[0].ConnectionManager = DtsConvert.ToConnectionManager90(flatfileConnectionManager);
managedInstance.AcquireConnections(null); managedInstance.ReinitializeMetaData(); managedInstance.ReleaseConnections(); // Get the destination's default input and virtual input. IDTSOutput90 output = sourceDataFlowComponent.OutputCollection[0]; // Here I dont find any columns at all..why?? // Now adding the data flow components IDTSComponentMetaData90 destinationDataFlowComponent = dataFlowTask.ComponentMetaDataCollection.New(); destinationDataFlowComponent.Name = "Destination Oledb compoenent"; // Here is the component class id for Oledvb data destinationDataFlowComponent.ComponentClassID = "{E2568105-9550-4F71-A638-B7FE42E66922}"; CManagedComponentWrapper managedOleInstance = destinationDataFlowComponent.Instantiate(); managedOleInstance.ProvideComponentProperties(); destinationDataFlowComponent. RuntimeConnectionCollection[0].ConnectionManagerID = oledbConnectionManager.ID; destinationDataFlowComponent. RuntimeConnectionCollection[0].ConnectionManager = DtsConvert.ToConnectionManager90(oledbConnectionManager); // Set the custom properties. managedOleInstance.SetComponentProperty("AccessMode", 2); managedOleInstance.SetComponentProperty("OpenRowset", "[MYDATABASE].[dbo].[MYTABLE]"); managedOleInstance.AcquireConnections(null); managedOleInstance.ReinitializeMetaData(); // Throws exception managedOleInstance.ReleaseConnections(); // Create the path. IDTSPath90 path = dataFlowTask.PathCollection.New(); path.AttachPathAndPropagateNotifications(sourceDataFlowComponent.OutputCollection[0], destinationDataFlowComponent.InputCollection[0]); // Get the destination's default input and virtual input. IDTSInput90 input = destinationDataFlowComponent.InputCollection[0]; IDTSVirtualInput90 vInput = input.GetVirtualInput(); // Iterate through the virtual input column collection. foreach (IDTSVirtualInputColumn90 vColumn in vInput.VirtualInputColumnCollection) { managedOleInstance.SetUsageType( input.ID, vInput, vColumn.LineageID, DTSUsageType.UT_READONLY); } DTSExecResult res = pkg.Execute(); } public ConnectionManager CreateOLEDBConnection(Package p) { ConnectionManager ConMgr; ConMgr = p.Connections.Add("OLEDB"); ConMgr.ConnectionString = "Data Source=VSTS;Initial Catalog=MYDATABASE;Provider=SQLNCLI;Integrated Security=SSPI;Auto Translate=false;"; ConMgr.Name = "SSIS Connection Manager for Oledb"; ConMgr.Description = "OLE DB connection to the Test database."; return ConMgr; } public ConnectionManager CreateFileConnection(Package p) { ConnectionManager connMgr; connMgr = p.Connections.Add("FLATFILE"); connMgr.ConnectionString = @"D:MyCSVFile.csv"; connMgr.Name = "SSIS Connection Manager for Files"; connMgr.Description = "Flat File connection"; connMgr.Properties["Format"].SetValue(connMgr, "Delimited"); connMgr.Properties["HeaderRowDelimiter"].SetValue(connMgr, Environment.NewLine); return connMgr; }
I have created an application that loads a package and executes it using DTS runtime classes. But when I run the application on a machine where only SQL Express edition is installed it's throwing
"Retreiving COM class factory for component with CLSID E44847F1-FD8C-4251-B5DA-B04BB22E236E failed due to the following error : 80040154"
I wanted to thank everyone for posting a ton of valuable information in these forums. I also want to thank all the moderators that have been replying with really insightful help!
I am trying to programmatically create an SSIS package to take .CSV data and put it into a SQL Server 2005. I am assuming that this is pretty common scenario.
I have used many of the examples in this forum as well as heavily borrowing from this example http://www.codeproject.com/csharp/Digging_SSIS_object_model.asp written by Moim Hossain.
I can get my package to create and execute properly but no data is being written to the SQL Server table. This has puzzled me for the last 2 days!
I know the issue isnt with the server itself because I tested it by graphically creating a test SSIS package and it transfers the .CSV data to the table perfectly.
Would anyone know why this would happen? The Execution results are returning success but no data is written to the table!
Could anyone please provide insight as to what my issue may be?
Thanks in advance!
Code Snippet
using System; using System.IO; using System.Data.SqlClient; using System.Collections.Generic; using System.Text; using Microsoft.SqlServer.Dts.Runtime; using PipeLineWrapper = Microsoft.SqlServer.Dts.Pipeline.Wrapper; using RuntimeWrapper = Microsoft.SqlServer.Dts.Runtime.Wrapper;
// Set some common properties of the connection manager object. //flatFileConnectionManager.Properties["ColumnNamesInFirstRow"].SetValue(flatFileConnectionManager, false); flatFileConnectionManager.Properties["Format"].SetValue(flatFileConnectionManager, "Delimited"); flatFileConnectionManager.Properties["TextQualifier"].SetValue(flatFileConnectionManager, """); flatFileConnectionManager.Properties["RowDelimiter"].SetValue(flatFileConnectionManager, " "); flatFileConnectionManager.Properties["DataRowsToSkip"].SetValue(flatFileConnectionManager, 0);
// Create the source columns into the connection manager. CreateSourceColumns(); }
private void CreateSourceColumns() { // Get the actual connection manager instance RuntimeWrapper.IDTSConnectionManagerFlatFile90 flatFileConnection = flatFileConnectionManager.InnerObject as RuntimeWrapper.IDTSConnectionManagerFlatFile90;
public class Column { private String name; private Microsoft.SqlServer.Dts.Runtime.Wrapper.DataType dataType; private int length; private int precision; private int scale; private int codePage = 0;
public String Name { get { return name; } set { name = value; } }
public Microsoft.SqlServer.Dts.Runtime.Wrapper.DataType DataType { get { return dataType; } set { dataType = value; } }
public int Length { get { return length; } set { length = value; } }
public int Precision { get { return precision; } set { precision = value; } }
public int Scale { get { return scale; } set { scale = value; } }
public int CodePage { get { return codePage; } set { codePage = value; } } }
String a = sourceComponent.RuntimeConnectionCollection[0].Name.ToString(); String b = sourceComponent.OutputCollection[0].Name; String c = sourceComponent.OutputCollection[0].Description; String d = sourceComponent.OutputCollection[0].OutputColumnCollection.Count.ToString();
// Create a path and attach the output of the source to the input of the destination. PipeLineWrapper.IDTSPath90 path = ((dataFlowTask as TaskHost).InnerObject as PipeLineWrapper.MainPipe).PathCollection.New(); path.AttachPathAndPropagateNotifications(sourceComponent.OutputCollection[0], destinationComponent.InputCollection[0]);
I am learning SSIS and I am working through the Wrox book on 2005 SSIS. When I try to Import the files from the AdventureWorks2000 database I cannot see the Human Resources files. I was able to go through the wizard once and see the files and select them and proceed, but now I cannot see them. Please keep in mind I am a newby to this. What should I check for or what am I doing wrong?
Is there anyone out there that can help me - please! I'm about to put my head through a wall.
I used to be able to create SSIS packages in VS 05. It's been a while since I've done it, but when I went in the other day, it would let me - all I can do is get errors. It tells me an "error prevented the view from loading" when it opens in control flow (blank packages and ones I created before all this started) and I get this error :
"Class not registered (Exception from HRESULT: 0x80040154 (REGDB_E_CLASSNOTREG)) (System.Windows.Forms)"
If I go over to the Data Flow tab and there are no errors until I click "Click here to add a new Data Flow task", then I get this :
"Object reference not set to an instance of an object. (Microsoft.DataTransformationServices.Design)"
So... here's what I've tried.
Uninstalled and reinstalled VS 05 and SP1 (this was before I realized the problem was with SQL).
Uninstalled and reinstalled Office 2003 Web Tools.
Made sure all the MSXMLs were registered.
Uninstalled and reinstalled all SQL Servers and Tools/Components.
Made sure all SQL Services were running under Local Account
Made sure Integrated Services service was running.
Applied all SQL patches.
I cannot get this to work and I need to be able to create packages. The only other step I can think of is wiping my computer and I don't want to have to do that.
Please please can anyone help me?? I've been scouring the internet and working on this for 1 day and 1/2.
Can someone point me to any whitepaper on how to use a Transact SQL script to create an SSIS package. I will need to be able to run the script at various customer sites.
I want to create a package that imports data from a Visual Foxpro database to SQL Server 2005 Express database. I used the wizard in BI Development Studio (similar to the DTS in SQL Server 2000) to create a package and noticed that the SQL statements created in the Preparation SQL Task only has code for creating tables. I want to make the package such that it first creates the destination database before creating all the related tables in it! When I tried to edit the SQL code to include DROP DATABASE and/or CREATE DATABASE statements, these were rejected.
Is it possible to do this or do I have to first create the database outside the package and then call the package? I want to make this a seamless process for clients who do not have the know-how of SQL Server database administration.
i have to manipulate/calculate an attribute at runtime in a number of dataset within a package. since the manipulation will be the same for all these datasets and that it has to be done with a script, i am looking to find a way of creating a single instance of this script and call it as and when necessary.
could i use a variable of data type object and pass and retrieve parameters into and out of it?
lets assume the following scenary to illustrate my point. i have to create a new attribute in two datasets ( ds1, ds2) which is based on the following attributes: firstname surname
and that i want to create this new attribute as:
surname, firstname
so i am creating a single string composed of surname plus a comma and firstname.
thus i will be creating a new attributes called custNames for two or more dataset and it tomorrow it is decided that i should be done in a different way then i only need to amend in on place.
i know it is not clear but i will be glad to anwser any queries to clarify this question.
I am new to SQL Server 2005, I need Help to create a SSIS package for the below taks, My task is Refer a Column [Status] in the Logtable which is used to maintain the logs for the running processes, When ever any Rows with the "FAIL" status in Status column occurs in the LogTable then get the Respective ErrorRowNumber and get the Details of the Error from other 2, 3 tables which will be a simple SQL statements, when you get the Error Details then mail these details to the given Email id.
For this I need a help In creating a SSIS package which will continuously check the Status and if any FAIL status occurs then the Error details need to mail to mail id which are stored in the config files of SSIS package.
If any one has better idea to send the mails then please let me know.
if any online tutorial is available that will also help. Thanks in advance
Create a SSIS package for following scenario.I have one excel file which will contain 10 records for Monday, 12 records for Tuesday, 7 on Wed, no records on Thursday so if records are there I get mail if no records are there I didn't get mail daily.
Hi, I want to create a package to import some tables from database X from Server XYZ to database X of server ABC. (As my X database on server XYZ is gets updating everyday so i need to update it on X of server ABC using the package.) So i have created a package using the import export data transformationn services. It runs fine while creating. i.e importing data for the first time. But when i have saved that SSIS package on SQL or File system and scheduled it to run daily, but if fails everytime. I am not getting the error its giving. Because everytime when i go to view history of that package it just gives me messages like step1 started by user xyz and failed. Can you please help me to sort out this problem. If possible give me steps which will help me to create package to run above scenario. you can mail me the solution on abhijeets@nedbank.co.za
I am receiving the following error when attempting to create a new package in SSIS 2005. I have completeley re-installed, and even had our support folks re-image my pc, but no luck. Works fine on my laptop, and others are not having any issues.
Microsoft Visual Studio is unable to load this document: Index (zero based) must be greater than or equal to zero and less than the size of the argument list.
Any help would be greatly appreciated. Thanks, Bryan
I am new to SSIS. I have a requirement that from a flat file need to import the data into SQL Server DB(SQl Server2008r2).
1.When the file doesn't found in dir need to send a mail. 2.error reading And writing ,on which record the error occurred and capture the error details and need to send mail. 3. In success also need to send a mail.
I am looking high and low for some assistance with developing a VB .NET solution that I programmatically create a package and add tasks. I am adding a BULK INSERT task to load large FLAT TEXT files into SQL Server 2005 tables. When I execute the application I execute a package validation and it always returns FAILURE. I have been reading and searching like crazy and I have bought 2 microsoft books, TO NO AVAIL! Can anyone PLEASE help me with this. Thank you!
I am trying to dynamically create the connection to a database within an SSIS package.
the requirement is to allow the user to pass through the database as a variable and that variable will dynamically create the connection string in the connection manager.
Requirement: When we create SSIS Pacakge using Businessinteligence studio.Each Source and Destination or whatever we using the Control required DB Connection.
we connect theDB server and Database Table through manaully .Instead of Manual i need dynamic Global varible for Connection String .How to achieve this connection string.
because suppose we create SSIS Package in Developement Server Latter We change the Server from Developement to Another Testing Server . at that time we dont requierd for changing manulay.any one pls reply me.
Same as in Dotnet we give configiration XML file .we gave the Connection strng. how to in SSIS we do?
2) NG20150623.txt 2015-06-23 00:00:00.000 NG 20150701 43
3) HO20150624.txt 2015-06-24 00:00:00.000 HO 20150701 43 And so on..
But the requirement is to have a dynamic query where we can have more number of Codes or less number of codes and similarly the package should generate dynamic text files, one .txt file per code. What is the best way to create a package which can meet the above requirement?
We are in the process of trying to automate our production releases (what a concept ;-)
The database is SQL server 2005 All objects are being stored in VSS Using Nant and Cruise Control for the actual migrations. I have two directories - Create (for a brandnew database) and Change (db object changes)
In my 'Change' script, I do the following -
1 - Take backup of database 2 - Migrate objects from 'change' directory to production 3 - Script out all objects of database and save in the 'Create' directory
For the #3, I was hoping I could create an SSIS package that would script out all database objects and save them on the VSS server.
I'm new to SSIS and want to verify it's something that can be done before I start down that path. If anyone has any examples or references, it would be much appreciated.
I have a SQL2000 DTS package that executes vbscript to loop through a recordset which:
- runs a stored procedure and populated tables
- builds a recordset from the populated tables to write records to an Excel file
- writes status to text files with either the error or success notices
I use FSO to set up the success and error files, but the scheduled job in SQL2005 which calls the SSIS package returns the following error:
"Retrieving the file name for a component failed with error code 0x0015F74C"
I can successullly run this (vbscript) in both the SSIS package via the BI Development Studio and in MS Access (exactly the same code in both) - but not as a SSIS package called in a scheduled job in SQL2005.
I am at an impasse with this ... any and ALL assistance would be GREATLY appreciated.
I'm not at all comfortable with SSIS so please forgive me if I overload you all with information here:
I need to create a data table using SSIS which does not delete the previous days data. So far all the data tables we use to write reports in Visual Studio are constructed in SSIS as follows.
1 - Excecute SQL Task - DELETE FROM STOCK 2 - Data Flow Task 3 - Data Reader Source - SELECT * FROM ODBCDATASOURCE 4 - OLE DB Destination (Creates table STOCK)
The data tables which are created this way are stored in a data warehouse and scheduled to refresh once a day, which means that any data from yesterday is lost when the updates run. So, I tried to create a table which never has its previous days' data deleted by using just the last three steps above - and it worked great in Visual Studio, no problem at all. However, when I added this SSIS Package to the Update Job in SQL Server Management Studio, the job totally rejected the packed with the message: "The command line parameters are invalid. The step failed".
I thought I could work around this problem by asking the job step to excecute a simple SQL query to insert the data from table1 into table2 (and would thus negate the need for a SSIS Packege at all), but it threw me a curve ball with some message about not being able to use proxy accounts to run T-SQL Scripts.
If anyone knows how to create a SSIS package in which the data never expires please could you impart some wisdom my way. I only need to do this once for a specific report.
Please, when answering, bear in mind that I'm a simple fellow with little understanding of the inner workings of SQL Server and its various components, so please use short sentences and simple words.
Hi, I have metadata that stored my table structure and relationship. I would like to know is it possible to create table relationship programatically? Any sample?
I need to be able to create a DSN through code that connects to SQL Server using SQL Native Client. I have found the following article: http://support.microsoft.com/kb/q184608/. This demonstrates most of what I need, but it does not show how to set the username and password, which is something that I need to do.
Can a package executed from code be aborted in code? The Package class has an Execute() method but no Abort() method. Clearly the debugger can stop a package at any point, so it must be possible somehow.
I built a small application that helps me create new packages by loading a template and modifying some parameters, and eventually save the updated package to a user defined location. This application uses the SSIS programming API.
I know all the packages created this way from the same template will have the same package ID. Is there a way to regenerate the package ID programmatically? I don't want to use dtutil.exe, but rather a nice package level method in the API.
hi my domain is hosted on a remote server related databases are stored on a seperated DataServer I have the name of that DataServer and i need to create new DBs programmatically
I use this statement on my local machine sqlstat= "CREATE DATABASE dbc_" + dbid + " ON PRIMARY" + "(Name=db_" + dbid + ", filename = '" & Server.MapPath("~/app_data") & "" + dbfilename + "')" but it will not work , because i dont know the physical path i cant use Server.MapPath()
