Problem: SSIS Package Failure Using 32-bit DTExec And 64-bit DTExec
Apr 17, 2008
Hi all,
I have a serious problem with my SSIS Package while executing using 32-bit DTExec and 64-bit DTExec.
Here are the details:
Environment:
Windows Server 2003 64-bit (Build 3790: Service Pack 2)
SSIS 32-bit & 64-bit installed
SQL Server 2005 (Microsoft SQL Server 2005 - 9.00.1399.06 (X64) - RTM)
SSIS Package details (compiled in 64 bit)
Script tasks only
Microsoft Visual Basic .NET (using TRY...CATCH block)
PreCompileScriptIntoBinaryCode = TRUE
Run64BitRunTime = TRUE
Execution
Batch file that uses DTExec to execute the Package.
SCENARIO
I am trying to exeucte the above SSIS package using both 32-bit and 64-bit DTExec to make it failure by providing invalid connection string. Here are the details,
Wrong connection String using 32-bit Execution
While establishing the connection the error message has been nicely captured in to my Exception block and writes into the log file.
Wrong connection String using 64-bit Execution
While establishing the connection the error has not been catpured anywhere (although I have TRY CATCH block) and it haults there itself with the message "Process is terminated due to StackOverflowException". Later I found that the error is due to the connection string along with the unhandled exception.
Please suggest any one of the following my findings, also if you have any other advice would be very much appreciated.
1. Shall I go ahead and fix the issue by handling those unhandled errors? (e.g Appdomain, application). I tried several but still not working using 64-bit DTExec.
2. Shall I go ahead and use 32-bit DTExec to execute the package? If so, is there any other major issue...like performance or anyother bug?
P.S: We cannot apply any service pack for SQL Server 2005 at the moment. Sorry abt it. If you have any specific hotfix for DTExec (without affecting SQL Server) then we can decide.
Sorry for the lengthy one and Thanks very much for you help in advance .
It looks like you can run an SSIS package via the DTEXEC command. You can also specifiy a configuraiton file flag on the command. Is there anyway to specify a configuraiton filter if you stored it into a table vs and xml file?
I developed my package in BI studio and tested runnning it through BI. Now when i have to deploy the package to main server it will be executed via batch job. This package requires an input file name too. Now when i am trying to run the package on my loacal box using following command on command prormpt i get error given below.
COMMAND USED BY ME: DTExec /FILE 'C:ProjectIntegrationServicesSFRIntegrationServicesinpkgSFRLocation.dtsx' /SET Package.Variables[InputParamFileName].Value;'C:DataDumpInputLOC00022'
ERROR MESSAGE: Microsoft (R) SQL Server Execute Package Utility Version 9.00.1399.06 for 32-bit Copyright (C) Microsoft Corp 1984-2005. All rights reserved.
Started: 1:29:54 PM Error: 2008-02-22 13:29:54.96 Code: 0xC0011007 Source: {9B1FAA88-A6FA-44CF-A2E2-DB3F737FDA65} Description: Unable to load the package as XML because of package does not ha ve a valid XML format. A specific XML parser error will be posted. End Error Error: 2008-02-22 13:29:54.96 Code: 0xC0011002 Source: {9B1FAA88-A6FA-44CF-A2E2-DB3F737FDA65} Description: Failed to open package file "'C:ProjectIntegrationServicesSFR IntegrationServicesinpkgSFRLocation.dtsx'" due to error 0x800700A1 "The speci fied path is invalid.". This happens when loading a package and the file cannot be opened or loaded correctly into the XML document. This can be the result of either providing an incorrect file name was specified when calling LoadPackage o r the XML file was specified and has an incorrect format. End Error Could not load package "'C:ProjectIntegrationServicesSFRIntegrationServices inpkgSFRLocation.dtsx'" because of error 0xC0011002. Description: Failed to open package file "'C:ProjectIntegrationServicesSFRInt egrationServicesinpkgSFRLocation.dtsx'" due to error 0x800700A1 "The specifie d path is invalid.". This happens when loading a package and the file cannot be opened or loaded correctly into the XML document. This can be the result of eit her providing an incorrect file name was specified when calling LoadPackage or t he XML file was specified and has an incorrect format. Source: {9B1FAA88-A6FA-44CF-A2E2-DB3F737FDA65} Started: 1:29:54 PM Finished: 1:29:54 PM Elapsed: 0.047 seconds
PLease let me know solution to creating batch job for executing this pkg with input param.
Hi, I have a package which has 2 file system tasks and 2 data flow tasks all in a for each loop container. if i execute the package from the development studio its executes fine but when i try to run it from the command prompt using the dtexec utility..it just runs upto the first data flow task and then it hangs unexpectedly. Any help on this would be really appreciated.
both variables as you can see are in the top level. the problem is that ChannelCode is string but Status is Integer. if I take out the /SET Package.Variables[Status].Value;1 part from the command line it will work fine. I think th eproblem is down to diffenrt types of variable as there are two types of String and Interger.
any ideas on what the problem migth be. p.s. I am runnning SQL server 2005 Dev edition (without any SP) Cheers
I created a package and stored in SQL server. I am running it as follows
master.dbo.xp_cmdshell dtexec /sq "ABC DEF GHI" /ser "Prod" ABC DEF GHI --> is my package name. When i run this from a job, it runs without error but when i run the above sql i am getting error as
Msg 102, Level 15, State 1, Line 2
Incorrect syntax near '/'. Am i missing some thing, please advice. Is that the problem between spaces in the package name?
Hi everybody. I'm trying to run a package, stored in MSDB, using dtexec utility from cmd. I do this using "dtexec /sq Main_Package /ser local /user sa /Password Oracle"
But all I got is: Microsoft (R) SQL Server Execute Package Utility Version 9.00.3042.00 for 32-bit Copyright (C) Microsoft Corp 1984-2005. All rights reserved. Started: 12:14:15 Could not load package "Main_Package" because of error 0xC0014062. Description: The LoadFromSQLServer method has encountered OLE DB error code 0x80004005 (Login timeout expired). The SQL statement that was issued has failed. Source: Started: 12:14:15 Finished: 12:14:34 Elapsed: 18.297 seconds
It runs perfectly from "Integration Services" directly(SQL Management Studio), but not from cmd. I tried to run it withuot login and password, the same result
I have a file-deployed package which I need to call N times at once using DTEXEC. When I attempt to execute N instances of the package, the second instance doesn't execute. I can execute different packages at the same time, but never the same package more than once. Does anyone know how to get this scenario working? Just to clarify a bit more - I can call the package as many times as I like as long as I wait until the previous run of DTEXEC is finished. Please help!
I am executing a package via vb.net with package.execute using the IDTSEvents interface inheriting DefaultEvents. I am trying to emulate DTExec functionality. The package runs fine and I am catching events, but I don't get nearly as many info and progress events as are put out by DTExec. From my understanding DTExec also used the managedDTS application object, is that not the case? If it uses this does anyone have an ideas on how it is grabbing more events? For example I don't get these events that DTExec shows ..
Info: 2006-09-26 14:22:27.97 Code: 0x40016041 Source: DWB02130 Description: The package is attempting to configure from the XML file "D:SSISConfigurationsDatabase.dtsConfig". End Info Info: 2006-09-26 14:22:28.05 Code: 0x40016040 Source: DWB02130 Description: The package is attempting to configure from SQL Server using the configuration string ""Configurations Database";"[dbo].[SSISConfigurations]";"DWB02130";". End Info
I also don't get as many progress updates during validation, I basically get 0 and 100% while DTExec shows several 1%, 3%, 10%, etc.
Here's is the situation : I have a script component that uses the aspnet membership provider to create users from a csv file in an aspnetdb database. In my Business Intelligence Developpement Studio (BIDS) everything works well. I added the aspnet membership provider in DtsDebugHost.exe.config and when I run the package from the BIDS it does what it have to do.
The problems begin when I deploy the package in file system as well as in sql server. When I run it doesn't use the DTExec.exe.config to find where are added the membership provider. If I refer to that post : http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=541086&SiteID=1 When I run the package from SQL Server it should use the DTExec.exe.config as an app.config for my script component. But it doesn't use it at all and stop with an error that have nothing to do with that. I'm sure that it doesn't use because even when I have a wrong DTExec.exe.config it shows the same error.
If some one knows something about that it will be really great. I need to force my package when it's in stored packages to use one of the *.config as an app.config file.
Hi, I am web developer. But for the last few weeks i have been working on SQL2005. I actually exported data from a table (SQL 2000) into the Excel sheet and then i have been trying to import data from Excel sheet into another table(SQL 2005). For this i had created SSIS package successfully. I had added this into integration services project(MicroSoft). After pressing CTRL+F5. I am repeatedly getting the following error DTExec:The Package Execution returned DTSER_FAILURE<1> Please help me because requirement is urgent.
I'm having an odd issue trying to run an SSIS package on a 64 Bit SQL 2005 SP2 (build 3042) Windows 2003 SP2 64 bit AMD Cluster. I have to use the dtexec cmd line utility from SQL Server Agent because I need to connect to an Access database. When running the package from dtexec, it doesn't seem to be able to load the package, i keep getting this error:
Executed as user: USSI-CORPsql02admin. ...00.3042.00 for 32-bit Copyright (C) Microsoft Corp 1984-2005. All rights reserved. Started: 1:56:00 PM Error: 2008-03-03 13:56:01.28 Code: 0xC0010018 Source: {DFBF51D9-DF81-41B9-9370-9DFC68E28FBA} Description: Error loading value "<DTSropertyExpression xmlnsTS="www.microsoft.com/SqlServer/Dts" DTS:Name="Subject">@[User::vString]</DTSropertyExpression>" from node "DTSropertyExpression". End Error Error: 2008-03-03 13:56:01.28 Code: 0xC0010018 Source: {DFBF51D9-DF81-41B9-9370-9DFC68E28FBA} Description: Error loading a task. The contact information for the task is "Send Mail Task; Microsoft Corporation; Microsoft SQL Server v9; ? 2004 Microsoft Corporation; All Rights Reserved;http://www.microsoft.com/sql/support/default.asp;1". This happens when loading a task fails. End Error Error: 2008-03-03 13:56:01.53 Code: 0xC001000E Source: TestMsgBox Description: The connection... Process Exit Code 1. The step failed.
The @[User::vString is a string variable in the package, and this particular package is actually just using the SendMail task without using an Access DB, as i'm trying to just get the dtexec utility to work. I believe i'm either doing something wrong or have found a bug, as i've used this in the past at another company without any issue.
Here's the cmd line that I'm using (I'm using the 32 bit version of dtexec): E:Progra~1Micros~190DTSBinndtexec.exe /SQL "TestMsgBox" /SERVER D1PSQL02V /CHECKPOINTING OFF /REPORTING EW
I want execute package with SQL Server Agent. I create a job and I try all the solutions : my job doesn't want to work.
I success to launch the package but the execution failed.
I try the same job launching from the server : it doesn't work either.
So, I copy the command line (/FILE, /DTS or /SQL) and with DTEXEC utility it works when this command is executed on the server (on a client computer, the problem "the product level is insufficient" appears).
Why is it working with this utility and not with a job ?
I try to change every secured option (credential --> proxy --> Run as ; change the owner of the job ; etc.)
We have a problem with visual studio. It hangs when i use the "execute package" option. New packages are running correctly but the packages which i've already built are not executing anymore...Any ideas to get things on track again?
I am finishing a port of a project that was done with RS2000/AS2000/DTS2000. The cube process is triggered remotely by a Korn Shell script from Services for Unix that runs a DTS package:
I have finished porting the AS and RS parts to their 2005 equivalents and now I am trying to get the cube processing converted. I have built a SSIS package that processes the cube and I have deployed it to the SQL Server in MSDB. When I am logged on to the SQL Server box directly, I can execute the package via the Management Studio and from the command line via:
When I try to register the SQL Server in Integration Services from my client PC Management Studio it fails with "Access Denied". When I try to run the dtexec from my client PC it fails with "Access Denied".
I have walked through Kirk Haselden's instructions at:
(adding myself to Distributed COM Users, noting the correct DCom Config settings for MsDtsServer and restarting the Integration Services service) and the problem is still not resolved. I have posted to the DTS group on Technet and now I am posting here. This can not be run scheduled by SQL Agent as it needs to be triggered by other load/postload activities. Are there any other fixes for this problem that I can try?
Thanks,
Keehan
By the way, being able to have all the parts of this project in one dev environment is great. The previous application was developed in 3 separate locations which was pretty clunky.
I have a package that executes fine in BIDS. However, when calling it using xp_cmdshell it cannot find the path to the source flat file. Anyone have any suggestions?
Microsoft (R) SQL Server Execute Package Utility Version 9.00.1399.06 for 32-bit Copyright (C) Microsoft Corp 1984-2005. All rights reserved. NULL Started: 9:50:56 AM Progress: 2006-02-10 09:50:57.40 Source: Data Flow Task Validating: 0% complete End Progress Progress: 2006-02-10 09:50:57.40 Source: Data Flow Task Validating: 50% complete End Progress Progress: 2006-02-10 09:50:57.42 Source: Data Flow Task Validating: 100% complete End Progress Progress: 2006-02-10 09:50:57.43 Source: Data Flow Task Validating: 0% complete End Progress Progress: 2006-02-10 09:50:57.43 Source: Data Flow Task Validating: 50% complete End Progress Progress: 2006-02-10 09:50:57.43 Source: Data Flow Task Validating: 100% complete End Progress Progress: 2006-02-10 09:50:57.43 Source: Data Flow Task Prepare for Execute: 0% complete End Progress Progress: 2006-02-10 09:50:57.43 Source: Data Flow Task Prepare for Execute: 50% complete End Progress Progress: 2006-02-10 09:50:57.43 Source: Data Flow Task Prepare for Execute: 100% complete End Progress Progress: 2006-02-10 09:50:57.43 Source: Data Flow Task Pre-Execute: 0% complete End Progress Warning: 2006-02-10 09:50:57.43 Code: 0x80070003 Source: Data Flow Task Source - NATNLACCTS [1] Description: The system cannot find the path specified. End Warning Error: 2006-02-10 09:50:57.43 Code: 0xC020200E Source: Data Flow Task Source - NATNLACCTS [1] Description: Cannot open the datafile "Q:BINATNLACCTS". End Error Error: 2006-02-10 09:50:57.43 Code: 0xC004701A Source: Data Flow Task DTS.Pipeline Description: component "Source - NATNLACCTS" (1) failed the pre-execute phase and returned error code 0xC020200E. End Error Progress: 2006-02-10 09:50:57.43 Source: Data Flow Task Pre-Execute: 50% complete End Progress Progress: 2006-02-10 09:50:57.43 Source: Data Flow Task Cleanup: 0% complete End Progress Progress: 2006-02-10 09:50:57.43 Source: Data Flow Task Cleanup: 50% complete End Progress Progress: 2006-02-10 09:50:57.43 Source: Data Flow Task Cleanup: 100% complete End Progress DTExec: The package execution returned DTSER_FAILURE (1). Started: 9:50:56 AM Finished: 9:50:57 AM Elapsed: 0.985 seconds NULL
I developed my package in BI studio and tested runnning it through BI. Now when i have to deploy the package to main server it will be executed via batch job. This package requires an input file name too. Now when i am trying to run the package on my loacal box using following command on command prormpt i get error given below.
COMMAND USED BY ME: DTExec /FILE 'C:ProjectIntegrationServicesSFRIntegrationServicesinpkgSFRLocation.dtsx' /SET Package.Variables[InputParamFileName].Value;'C:DataDumpInputLOC00022'
ERROR MESSAGE: Microsoft (R) SQL Server Execute Package Utility Version 9.00.1399.06 for 32-bit Copyright (C) Microsoft Corp 1984-2005. All rights reserved.
Started: 1:29:54 PM Error: 2008-02-22 13:29:54.96 Code: 0xC0011007 Source: {9B1FAA88-A6FA-44CF-A2E2-DB3F737FDA65} Description: Unable to load the package as XML because of package does not ha ve a valid XML format. A specific XML parser error will be posted. End Error Error: 2008-02-22 13:29:54.96 Code: 0xC0011002 Source: {9B1FAA88-A6FA-44CF-A2E2-DB3F737FDA65} Description: Failed to open package file "'C:ProjectIntegrationServicesSFR IntegrationServicesinpkgSFRLocation.dtsx'" due to error 0x800700A1 "The speci fied path is invalid.". This happens when loading a package and the file cannot be opened or loaded correctly into the XML document. This can be the result of either providing an incorrect file name was specified when calling LoadPackage o r the XML file was specified and has an incorrect format. End Error Could not load package "'C:ProjectIntegrationServicesSFRIntegrationServices inpkgSFRLocation.dtsx'" because of error 0xC0011002. Description: Failed to open package file "'C:ProjectIntegrationServicesSFRInt egrationServicesinpkgSFRLocation.dtsx'" due to error 0x800700A1 "The specifie d path is invalid.". This happens when loading a package and the file cannot be opened or loaded correctly into the XML document. This can be the result of eit her providing an incorrect file name was specified when calling LoadPackage or t he XML file was specified and has an incorrect format. Source: {9B1FAA88-A6FA-44CF-A2E2-DB3F737FDA65} Started: 1:29:54 PM Finished: 1:29:54 PM Elapsed: 0.047 seconds
PLease let me know solution to creating batch job for executing this pkg with input param.
I've worked for awhile to get an SSIS package that connects to Oracle to succesfully run via DTExecUI from a 64 bit SQL Server. I completely understand that DTExecUI runs in 32 bit mode, but I need to initiate the package from a stored proc so I need to be able to utilize the 32 bit DTExec from either a xp_cmdshell call or from an sp_start_job (thanks to MatthewRouche for the suggestion). Both Ora92 and 10g were installed on the SQL Server.
Here's what I've tried unssuccessfully
1. SQL Job - Just calling the package from an SSIS step. I wouldn't expect this to run because it is 64 bit DTExec. 2. SQL Job - A job that calls the 32 bit version of dtexec from cmdexec. Here is 2 examples I have tried, the first is because I came across several postings saying that there is a problem with the "(x86)" for the Oracle driver.
\server_nameE$ProgramFilesLegacyMicrosoftSQLServerX8690DTSBinnDTExec.exe /DTS "PackageName" /SERVER "SERVER_NAME" /SET Package.Variables[PLT].Value;"8H" /CONFIGFILE "\SERVERConfigFile.dtsConfig" /MAXCONCURRENT " -1 " /CHECKPOINTING OFF /REPORTING EW
\server_nameE$Program FilesMicrosoft SQL Server (x86)90DTSBinnDTExec.exe /DTS "PackageName" /SERVER "SERVER_NAME" /SET Package.Variables[PLT].Value;"8H" /CONFIGFILE "\SERVERConfigFile.dtsConfig" /MAXCONCURRENT " -1 " /CHECKPOINTING OFF /REPORTING EW
Both of these give me the error below:
System.Exception: System.Data.OracleClient requires Oracle client software version 8.1.7 or greater.
I'm assuming that it is running in 32 bit mode because this was at the beginning of the job history log:
Executed as user: "XXX". ....3042.00 for 32-bit Copyright (C) Microsoft Corp 1984-2005. All rights reserved
3. I then tried to call the package from a batch file on a file server that had integration services installed on. Got the same error as above.
I'm obviously missing something if I can get it to work in DTExecUI but not DTExec 32-bit.
Any ideas? Below are some postings I have already visited several times:
Windows 64 bit and Oracle http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=358365&SiteID=1
Connecting to Oracle on 64-bit (x64) machine http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=159581&SiteID=1
One paragraph that stands out on this posting is:
Find the location of the application that is generating the error. Check the path to this location and see if it contains any parenthesis. If so, you must relocate the application to a directory without any parenthesis in the path. This didn€™t work for me, unless I was doing it wrong.
Error retrieving numeric data from Oracle 10g on 64 bit Itanium SSIS server http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=1183296&SiteID=1
Connection Problems to Oracle Database http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=1307460&SiteID=1
If this posting was complete, it looks like what I was looking for.
I'm running several SSIS-packages in stored procedure using dtexec. Actually, there is a software that is running the procedure, but that's not important here.
The problem: if some of the packages failes, the whole procedure does not fail and I don't know if all the packages are successfully completed.
Here is a sample list of ssis-packages from procedure:
1) We are migrating data from SQL Server 2000 to 2005 DB.
2) We are developing SSIS packages in client, build, deploy and install and copying to SQL Server 2005 server to execute for migration.
3) For this we encrypt all packages with password, then decrypt all while executing in a batch file using dtexec. This works fine.
4) What does not work is executing a master execute task in which are individual execute tasks. This is so since the other packages are still encrypted.
5) We decrypted them separately first using dtutil using Decrypt and Copy/Move option but while executing error is thrown that these other packages are still protected by password.
6) Do we need to decrypt, then encrypt using default package protection level and then decrypt again.
7) How exactly to just execute a master package when packages are developed on client.
Running eval. edition of Sql Server Standard 2005. "Insufficient product level" error is thrown during validation phase of an OleDBCommand data flow task. This task type is not licensed in Sql Server 2005 standard? The component runs a very simple sql update statement against a one row table in Sql Server 2005.
If it works from BIDS, should it not work from dtexec.exe on the same box?
Does dtexec run under the security context of the logged in user?
We made SSIS package in dev environment in windows 2008R2 and SQL server 2012. Same packages were placed on SAN disks in cluster environment and are invoked with security contexxt of admin user and wih dtexec utility ( we call this using an sp). rarely but procedure completed i see the log text file by SSIS package and find the return code 5 means package is unable to load. i canst found what are the exact reasons for return code 5.
In one of my SSIS package (which connect to DB2) i save the DB2 username and password in SSIS packge itself by using option "EncryptAllWithPassword". When I try to run the packge using dtexecui.exe it works perfectly fine. It prompts me for package password and then run fine.
Now I am trying to run it using command line dtxece.exe and i get fooolwoing error. Please suggest and help!!! DTExec.exe /FILE C:pkgDB2Load.dtsx /CONFIGFILE C:ConfigpkgConfigMaster.dtsConfig /Decrypt mypassword /CHECKPOINTING OFF /REPORTING EWCDI
Error Message: ---------------------- Description: An OLE DB error has occurred. Error code: 0x80040E21. An OLE DB record is available. Source: "IBM OLE DB Provider for DB2 Servers" Hresult: 0x80040E21 Description: " SQL30082N Attempt to establish connection fa iled with security reason "17" ("UNSUPPORTED FUNCTION"). SQLSTATE=08001 "
Our SSIS packages use the Web Service Task to call services to send email and write package failure data to a department wide database. These Web Service Calls are failing with HTTP 401 errors. It was caused by the passwords for the HTTP connections not being saved when the SSIS pkgs were saved to .dtsx files. I have tried saving the package with a password and the EncryptSensitiveWithPassword protection option. This password can be supplied when the package is called from another package or a package is executed in Visual Studio but cannot be supplied to DTexec to execute the package in a job. DTexec does have a /Password parameter but it is rejected if the package is loaded from a .dtsx file. This appears to be a bug in DTexec. It only accepts the /Password parameter when the package is loaded with the /SQL option. Specifying /Password and /File is not supported.
Is this a known bug? Are there any workarounds? Has anyone successfully called a Web Service from SSIS executed via DTexec?
I have a simple package in SSIS that reads approx 2.5million rows from SQL2k5 source and exports it into 4 Foxpro files. The way I do it is as below 1) use and oledb source to read data from SQL2k5 2) use a conditional split component to divide the source into 4 streams ( I do a recId % 4 = 0, 1, 2, 3 to give 4 streams) 3) use an OLEDB destination to export to the 4 prexisting foxpro tables.
This thing runs amazingly fast when I run(F5) it from Business Inteligence studio, but it is atleast 10 times slower when rightclick the package and execute using dtexec.exe
I can't figure out why! any clues??
I would elimiate the 64 bit /32 bit problems because export to foxpro only works in 32 bit mode and my package has been configured to run in 32 bit mode.
I have stumbled on a problem with running a large number of SSIS packages in parallel, using the €œdtexec€? command from inside an SQL Server job.
I€™ve described the environment, the goal and the problem below. Sorry if it€™s a bit too long, but I tried to be as clear as possible.
The environment: Windows Server 2003 Enterprise x64 Edition, SQL Server 2005 32bit Enterprise Edition SP2.
The goal: We have a large number of text files that we€™re loading into a staging area of a data warehouse (based on SQL Server 2k5, as said above).
We have one €œmain€? SSIS package that takes a list of files to load from an XML file, loops through that list and for each file in the list starts an SSIS package by using €œdtexec€? command. The command is started asynchronously by using system.diagnostics.process.start() method. This means that a large number of SSIS packages are started in parallel. These packages perform the actual loading (with BULK insert).
I have successfully run the loading process from the command prompt (using the dtexec command to start the main package) a number of times.
In order to move the loading to a production environment and schedule it, we have set up an SQL Server Agent job. We€™ve created a proxy user with the necessary rights (the same user that runs the job from command prompt), created an the SQL Agent job (there is one step of type €œcmdexec€? that runs the €œmain€? SSIS package with the €œdtexec€? command).
If the input XML file for the main package contains a small number of files (for example 10), the SQL Server Agent job works fine €“ the SSIS packages are started in parallel and they finish work successfully.
The problem: When the number of the concurrently started SSIS packages gets too big, the packages start to fail. When a large number of SSIS package executions are already taking place, the new dtexec commands fail after 0 seconds of work with an empty error message.
Please bear in mind that the same loading still works perfectly from command prompt on the same server with the same user. It only fails when run from the SQL Agent Job.
I€™ve tried to understand the limit, when do the packages start to fail, and I believe that the threshold is 80 parallel executions (I understand that it might not be desirable to start so many SSIS packages at once, but I€™d like to do it despite this).
Additional information:
The dtexec utility provides an error message where the package variables are shown and the fact that the package ran 0 seconds, but the €œMessage€? is empty (€œMessage: €œ). Turning the logging on in all the packages does not provide an error message either, just a lot of run-time information. The try-catch block around the process.start() script in the main package€™s script task also does not reveal any errors. I€™ve increased the €œmax worker threads€? number for the cmdexec subsystem in the msdb.dbo.syssubsystems table to a safely high number and restarted the SQL Server, but this had no effect either.
The request:
Can anyone give ideas what could be the cause of the problem? If you have any ideas about how to further debug the problem, they are also very welcome. Thanks in advance!
I have a command line as following, with DTEXEC to launch the execution of a package and to set a value (13335) of an user variable called CIB (string type):
But I have got an error message saying that the object is not known in the package. My variable does exist in the variable window of the dataflow part. Thank you for telling me what to set so that the variable can be set by the command line.
