XML Data Source .. Expression? Variable? Connection? Error: Unable To Read The XML Data.

Feb 23, 2008

RE: XML Data source .. Expression? Variable? Connection? Error: unable to read the XML data.

I want my XML Data source to be an expression as i will be looping through a directory of xml files.

I don't see the expression property or the connection property??

I tried setting the XMLData property to @[User::filename], but that results in:

Information: 0x40043006 at Load XML Files, DTS.Pipeline: Prepare for Execute phase is beginning.
Error: 0xC02090D0 at Load XML Files, XML Source [108]: The component "XML Source" (108) was unable to read the XML data.
Error: 0xC0047019 at Load XML Files, DTS.Pipeline: component "XML Source" (108) failed the prepare phase and returned error code 0xC02090D0.
Information: 0x4004300B at Load XML Files, DTS.Pipeline: "component "OLE DB Destination" (341)" wrote 0 rows.
Task failed: Load XML Files
Information: 0xC002F30E at Bad, File System Task: File or directory "d:jcpxmlLoadjcp2.xml.bad" was deleted.
Warning: 0x80019002 at Package: The Execution method succeeded, but the number of errors raised (2) reached the maximum allowed (1); resulting in failure. This occurs when the number of errors reaches the number specified in MaximumErrorCount. Change the MaximumErrorCount or fix the errors.
SSIS package "Package.dtsx" finished: Failure.
The program '[3312] Package.dtsx: DTS' has exited with code 0 (0x0).


Thanks for any help or information.

View 3 Replies


ADVERTISEMENT

Pipeline Error-excel Source-data Reader Does Not Read In Meta Data

Apr 16, 2008

Hi all, i got this error:


[DTS.Pipeline] Error: "component "Excel Source" (1)" failed validation and returned validation status "VS_NEEDSNEWMETADATA".

and also this:

[Excel Source [1]] Warning: The external metadata column collection is out of synchronization with the data source columns. The column "Fiscal Week" needs to be updated in the external metadata column collection. The column "Fiscal Year" needs to be updated in the external metadata column collection. The column "1st level" needs to be added to the external metadata column collection. The column "2nd level" needs to be added to the external metadata column collection. The column "3rd level" needs to be added to the external metadata column collection. The "external metadata column "1st Level" (16745)" needs to be removed from the external metadata column collection. The "external metadata column "3rd Level" (16609)" needs to be removed from the external metadata column collection. The "external metadata column "2nd Level" (16272)" needs to be removed from the external metadata column collection.


I tried going data flow->excel connection->advanced editor for excel source-> input and output properties and tried to refresh the columns affected.
It seems that somehow the 3 columns are not read in from the source file?
ans alslo fiscal year, fiscal week is not set up up properly in my data destination?
anyone faced such errors before?

Thanks

View 13 Replies View Related

Can I Set Excel Connection Manager's Data Source As A Variable?

Feb 1, 2006

I do not know the Excel file name to load in design time.

Would like to pass the value to a variable in the package in run time?



How to do this?



Thanks,



Guangming

View 3 Replies View Related

How Can Pass Variable Or Parameter In DATA READER SOURCE Ising ADO:NET Connection Manager

May 3, 2007

In SSIS in Sql task we have option to pass parameter or variable..But in Data Flow Task when we use Data Reader Source using ADO.NET connection..There is no option to pass parameter or variable Or no option to receive a parameter or variable .

I am having a query were it need to pass a parameter.in sql task ...And Data Reader Source have to receive this parameter from sql task .



Sql Task finds a value of parameter and pass to DataReader Source in DataFlow Task .. ...



Please can any one help me to solve this problem of Receiving parameter or variable in DataReader Source using DAO.Net connection in DataFlow Task..thank you dilsa

View 3 Replies View Related

Error: Unable To Retrieve Column Information From The Data Source

Jul 7, 2006

Hi,

I am trying to set up a data flow task. The source is "SQL Command" which is
a stored procedure. The proc has a few temp tables that it outputs the final
resultset from. When I hit preview in the ole db source editor, I see the
right output. When I select the "Columns" tab on the right, the "Available
External Column List" is empty. Why don't the column names appear? What is
the work around to get the column mappings to work b/w source and
destination in this scenario.


In DTS previously, you could "fool" the package by first compiling the
stored procedure with hardcoded column names and dummy values, creating and
saving the package and finally changing the procedure back to the actual
output. As long as the columns remained the same, all would work.
Thats not working for me in SSIS.

Thanks in advance.
Asim.

View 9 Replies View Related

Cannot Create A Connection To The Data Source Error

Apr 22, 2008



Hi,

I have setup SSRS with 3 data models using 3 different datasources on one server. I can create reports ok, but if any of the other users I want to access these reports tries to open the report I created or run there own reports (they can create a report), it gives them the error:

Report Execution Error

Cannot create a connection to the data source 'datasource1'

I do not recieve this error, but the other users do. I have set the other users as System administrator in the site-wide permissions, so they have access to everything.

Everything is set to integrated windows authentication.

On the Reporting Server Configuration Manager, the Unattended Execution Account is not selected and is blank.

What could cause these users not to be able to run existing or new reports they create?

Thanks for any help,

Steve

View 1 Replies View Related

An Error Has Occurred During Report Processing. A Data Source Instance Has Not Been Supplied For The Data Source DetailDS_get_o

Mar 13, 2008

hi ,

i am trying for a drill through report (rdlc)

ihave written the following code in drill through event of reportviewer, whenever i click on the first report iam getting the error like

An error has occurred during report processing.


A data source instance has no
t been supplied for the data source "DetailDS_get_orderdetail".







the code is



using System;

using System.Data;

using System.Data.SqlClient;

using System.Configuration;

using System.Collections;

using System.Web;

using System.Web.Security;

using System.Web.UI;

using System.Web.UI.WebControls;

using System.Web.UI.WebControls.WebParts;

using System.Web.UI.HtmlControls;

//using Microsoft.ApplicationBlocks.Data;

using Microsoft.Reporting.WebForms;

using DAC;

public partial class _Default : System.Web.UI.Page

{

protected void Page_Load(object sender, EventArgs e)

{

ReportViewer1.Visible = false;

}

protected void Button1_Click(object sender, EventArgs e)

{

DAC.clsReportsWoman obj = new clsReportsWoman();

DataSet ds = new DataSet();

ds = obj.get_order();

ReportViewer1.LocalReport.DataSources.Clear();

ReportDataSource reds = new ReportDataSource("DataSet1_get_order", ds.Tables[0]);



ReportViewer1.LocalReport.DataSources.Add(reds);

ReportViewer1.LocalReport.ReportPath = "C:/Documents and Settings/km63096/My Documents/Visual Studio 2005/WebSites/drillthrurep/Report.rdlc";

ReportViewer1.LocalReport.Refresh();

ReportViewer1.Visible = true;

}

protected void ReportViewer1_Drillthrough(object sender, DrillthroughEventArgs e)

{

DAC.clsReportsWoman obj = new clsReportsWoman();

ReportParameterInfoCollection DrillThroughValues =

e.Report.GetParameters();



foreach (ReportParameterInfo d in DrillThroughValues)

{

Label1.Text = d.Values[0].ToString().Trim();

}

LocalReport localreport = (LocalReport)e.Report;

string order_id = Label1.Text;

DataSet ds = new DataSet();

ds = obj.get_orderdetail(order_id);



ReportViewer1.LocalReport.DataSources.Clear();

ReportDataSource reds = new ReportDataSource("DetailDS_get_orderdetail", ds.Tables[0]);

ReportViewer1.LocalReport.DataSources.Add(reds);

ReportViewer1.LocalReport.ReportPath = Server.MapPath(@"Reportlevel1.rdlc");

ReportViewer1.LocalReport.Refresh();





}



}

the code in method get_orderdetail(order_id) is

public DataSet get_orderdetail(string order_id)
{
SqlCommand cmd = new SqlCommand();
DataSet ds = new DataSet();
cmd.Parameters.Add("@order_id", SqlDbType.VarChar, 50);
cmd.Parameters["@order_id"].Value = order_id;
ds = SQLHelper.ExecuteAdapter(cmd, CommandType.StoredProcedure, "dbo.get_orderdetail");
return (ds);
}pls help me.

View 1 Replies View Related

Connection Failed Error Creating Data Source For Server

Mar 1, 2008

Hopefully there's an easy answer for this. I'm totally new to SQL server and I'm trying to upsize an Access database. Whether I use the upsizing wizard or try to create a data source through the ODBC Data Source Administrator, I get a connection failed error. It tells me the server does not exist or my login is incorrect. I'm using NT authentication to connect to the server on my local drive. I just installed the whole 2005 express edition and the only thing that came up during the install was that I didn't have IIS installed. I didn't see how that would be relevant, so I went ahead with the install.

Shouldn't this work with the default setup? I've searched around trying different "solutions" although I doubt they were specifically related to this very basic problem. I tried giving my login every right I could find through in the database engine properties in the server management studio, which connects fine itself with just my network login name and no password. What am I missing?

View 3 Replies View Related

How Do I Add An ODBC Connection Data Source As A Data Flow Source

Mar 2, 2007

I have set up a new connection as a connection from data source, but I cannot see how to use this connection to create my Data Flow Source. I have tried using an OLE DB connection, but this is painfully slow! The process of loading 10,000 rows takes 14 - 15 minutes. The same process in Access using SQL on a linked table via DSN takes 45 seconds.

Have I missed something in my set up of the OLE DB source / connection? Will a DSN source be faster?

Thanks in advance

ADG

View 2 Replies View Related

Data Source Expression Not Working

Jun 6, 2007

Hello, I am trying to use a data source expression but I'm getting an error. Here is my data source expression:



="data source=" & Parameters!ServerName.Value & ";initial catalog=AdventureWorks"



I have created a report parameter called ServerName. However, when I deploy the report and report viewer opens, I get the following error:



An error has occurred during report processing. (rsProcessingAborted) Error during processing of the ConnectString expression of datasource €˜AdventureWorks€™. (rsDataSourceConnectStringProcessingError)



Is there some really obvious step I am missing here? The relevant parts of my rdl are below.



Thanks in advance



<DataSources>

<DataSource Name="AdventureWorks">

<ConnectionProperties>

<ConnectString>="data source=" &amp; Parameters!ServerName.Value &amp; ";initial catalog=AdventureWorks"</ConnectString>

<DataProvider>SQL</DataProvider>

</ConnectionProperties>

<rdataSourceID>(some stuff here)</rdataSourceID>

</DataSource>

</DataSources>

...

<ReportParameters>

...

<ReportParameter Name="ServerName">

<DataType>String</DataType>

<Prompt>ServerName</Prompt>

</ReportParameter>

</ReportParameters>

View 3 Replies View Related

Reporting Services :: Access Denied Error When Testing Data Source Connection In SSRS 2008

Aug 16, 2011

We are running Reporting Services 2008 R2 on a Windows Server 2008 Standard 64-Bit server.  I have a user that has full access to Reporting Services at all folder levels but IS NOT a local administrator on the 2008 server.

This user can create data source connections but when he tries to test the connection by clicking on the 'Test Connection' button, he gets the following error "The permissions granted to user <username> are insufficient for performing this operation.A user that has administrator priveleges on the server can test the connection fine. 

I don't want to make this user an administrator on the server. 

This is an extract from the log file:

ibrary!ReportServer_0-24!3478!08/16/2011-13:45:37:: Call to TestConnectForDataSourceDefinitionAction().
library!ReportServer_0-24!3478!08/16/2011-13:45:37:: e ERROR: Throwing Microsoft.ReportingServices.Diagnostics.Utilities.AccessDeniedException: , Microsoft.ReportingServices.Diagnostics.Utilities.AccessDeniedException: The permissions granted to user <username>
are insufficient for performing this operation.;

[code]....

View 10 Replies View Related

Cannot Create A Connection To Data Source 'data Source Name'

Dec 11, 2007

Today I was making a few reports.
When I tested the reports in Visual Studio, they worked great: I got the expected result.
But when I deployed the reports to our reportserver the problem started.
When I click on the directory in which my reports are deployed, I got my 4 reports.
Till now everything worked correct.
But when I click on a report to view the results it went wrong.
I got an error:
"Cannot create a connection to data source 'Live'"
(Live is the name of our data source).

We are using the Windows Logons and I am sure that I have all the rights on the server, I gave myself 'sysadmin' rights, so it should work.
I also have tried it with all the roles assigned on my account, but then it still won't work.

When I modify the data source, and set it to another server en database it works.
The datasource 'Live' exists on a x64 MsSQL server, en the other datasource is on a x86 MsSQL server.
Maybe that is the problem?

Can someone tell me what is wrong?

View 1 Replies View Related

SQL 2012 :: SSRS Data Shared Data Source Connection Login

Jan 12, 2015

Was wondering if there was a best practice minimum permissions for creating a SQL login to use when setting up a new shared Data source for SSRS report manager?

Something along the lines of them being a data read for the DB and permissions to update tempdb?

Would have thought it not advisable to have the login be able to update the main db...

View 1 Replies View Related

How To Get Read Only Control Value Through SQl Data Source

Nov 7, 2006

Hello,
I am using this sql data source control.
<asp:textbox runat="server" id="txtFromDate" ReadOnly="true"></></asp:textbox>
 
<asp:textbox runat="server" id="txtToDate" ReadOnly="true"></asp:textbox>
--------------------------------->
<asp:sqldatasource id="dsClickInfo" runat="server" connectionstring="<%$ ConnectionStrings:activeConnectionString %>"
selectcommand="SProc_GetTransaction" selectcommandtype="StoredProcedure">
<SelectParameters>
<asp:QueryStringParameter QueryStringField="Id" Name="MerchantID" Type="int32" />
<asp:ControlParameter ControlID="txtFromDate" Name="StartDate" PropertyName="Text"
Type="String" DefaultValue="0" />
<asp:ControlParameter ControlID="txtToDate" Name="EndDate" PropertyName="Text"
Type="String" DefaultValue="0" />
</SelectParameters>
 
</asp:sqldatasource>
I abouve code txtFromDate and txtToDate are marked as readonly so sqldatasource is not able to get value from these controls.
How is it possible?
Please help me..

View 3 Replies View Related

Expression-based Data Source + Subscription = No-go?

Dec 20, 2006

Set up reports to use expression-based data sources and everything works fine as far as executing and viewing the reports. The ConnectionString expression includes the credentials, and the "No credentials required" option is selected.

Unfortunately, am getting this when trying to create a subscription for same report. WTF?

eportingServicesService!processing!3!12/20/2006-12:20:07:: e ERROR: Throwing Microsoft.ReportingServices.ReportProcessing.ReportProcessingException: Cannot create a connection to data source 'XXXData'., ;
Info: Microsoft.ReportingServices.ReportProcessing.ReportProcessingException: Cannot create a connection to data source 'XXXData'. ---> System.InvalidOperationException: The ConnectionString property has not been initialized.
at System.Data.SqlClient.SqlConnection.PermissionDemand()
at System.Data.SqlClient.SqlConnectionFactory.PermissionDemand(DbConnection outerConnection)
at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
at System.Data.SqlClient.SqlConnection.Open()
at Microsoft.ReportingServices.DataExtensions.SqlConnectionWrapper.Open()
at Microsoft.ReportingServices.ReportProcessing.ReportProcessing.ReportRuntimeDataSourceNode.OpenConnection(DataSource dataSourceObj, ReportProcessingContext pc)
--- End of inner exception stack trace ---

View 15 Replies View Related

Variable Data Type In An Expression

Jun 7, 2006

Greetings my SSIS friends

I am attempting to create an expression as follows:



"Select * from someTable where someColumn >= " + (dt_str, 25, 1252) @[User::DateTimeVariable]



The problem is that my variable is a Datetime field and when I convert it to string, the string will not execute correctly.



How to solve this problem?

View 3 Replies View Related

SQL Server 2005 Unable To Read Image Data

Nov 27, 2006

This just happened today and I dont know why.  I have a database on a SQL server 2005 database, that has a table called photos.  It contains image data, but not the column says <Unable to read data>.  But before that I get an error when I try to execute the table to retreive the data.  The error is below. SQL Execution Error:Executed SQL statement: SELECT ID, AlbumID, Caption, BytesOriginal, BytesFull, BytesThumb FROM PhotosError Source: System.DataError Message: Invalid attempt to Read when reader is closed. This happens with SQL Server Management Studio, and Visual Studio 2005 when I try to access it.  I tried two computers so its not that.  Any help or insight would be appreciated. 

View 2 Replies View Related

Read Source Data Without Waiting For Possible Locks

Mar 16, 2007

hi!

I wonder if anyone can tell me how we can run select queries in an OLE DB data flow task and tell the target SQL 2000 server it should allow reads at all time. Currently when a lock is on the source table our SSIS package will sit and wait untill the lock on the source table is gone.

Thanks.

Marc

View 3 Replies View Related

Transferring Data From Read-only Source To Destination.

Oct 17, 2006

Hi,

I have read only permission in the source (OLTP) database. The source database is running in SQL Server 2000 and the size is more than 200 GB. I need to pull data from source and load target which is running in SQL server 2005.

Following are the objectives I want to achieve.



Data should be loaded on incremental basis.

Whatever changes take place (Update/Delete) in source, that should be replicated to already uploaded data.

Here I want to mention that, the source database does not have any identification key or timestamp column like Updated_Date by which I can filter the data which are recently inserted or updated into the source and upload the same. The source does not maintain any history data also. So I do not have any track of deleted record also.

I don€™t have any scope to change the schema in the source. In this scenario can anybody suggest me the best approach to achieve the above mentioned objectives?

Can I retrieve only the recent updated or inserted date form transaction log back up. Can log shipping solve the give the solution?

One more question. Say I have a table and I am exporting/importing all the data from/to my target table using SSIS or DTS. In this scenario does using query or using directly the table affects the performance?

Regards

Sudripta Rakshit

View 4 Replies View Related

Data Access :: Arithmetic Overflow Error Converting Expression To Data Type Int

Jul 24, 2015

When I execute the below stored procedure I get the error that "Arithmetic overflow error converting expression to data type int".

USE [FileSharing]
GO
/****** Object: StoredProcedure [dbo].[xlaAFSsp_reports] Script Date: 24.07.2015 17:04:10 ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO

[Code] .....

Msg 8115, Level 16, State 2, Procedure xlaAFSsp_reports, Line 25
Arithmetic overflow error converting expression to data type int.
The statement has been terminated.
(1 row(s) affected)

View 10 Replies View Related

Changing Data Types Of A Variable In An Expression

May 18, 2006

Greetings once again SQL friends,

Sorry to bug you with my silly questions but this little problem has been driving me crazy for the last hour or so!

I am attempting to change the data type of a variable in an expression but I keep getting an error saying that the expression can not be evaluated.

The syntax I am using is as follows :

(dt_str) @[User::MAX_OFFER_PRICE_ID]

My variable MAX_OFFER_PRICE_ID is type int32 and I am attempting to cast it as a string.

What am I doing wrong?!

 

Thanks for your help in advance.

View 7 Replies View Related

Is It Possible To Read Flat File Data From A Variable?

Nov 6, 2006

Hi,

I'm relatively new to SSIS and I've been tinkering with the XML and Flat File sources.

I noticed that in the XML source it is possible to tell SSIS to read the XML data from a variable. I didn't see a similar option for the Flat File source.

Does anyone know if it is possible to read flat file data from a variable when using the Flat File source?

Thanks,

-dhideal

View 5 Replies View Related

SSIS Data Flow Source Component To 'read' A PDF File

Feb 13, 2008

At our business we are getting a lot of PDF documents that are being hand keyed into a database. Has anyone heard ior know of a SSIS Data Flow Source component that I coud use to read thos documents into a data stream (?) and process?

View 5 Replies View Related

Read Single Piece Of Data From SqlDataSource To A Variable?

Jan 17, 2006

First I want to say thanks to the folks how have helped me over the past week or two.
I'm missing a crucial piece of insight. How do I read a single piece of data from a SqlDataSource which only hold one row of data into a declared variable?
string zUserName = SqlDataSource.(get the value in the row that is labeled UserName)
It really doesn't matter to me whether it is an SqlDataSource or some other data container that hold the data I've selected. My goal is to be able to read some data from some predefined datasource into a declared variable.
If I am approaching this problem in the wrong way any advice or article links you could give me would be grealy appreciate.
Thanks, Bill

View 1 Replies View Related

Unable To Create A New Data Source Using SQL Server.

Feb 24, 2007

I have recently installed Microsoft SQL Server 2005 Express. The installation seemed to go OK (Dell Optiplex I86, Win2K).

The SQL Server Management Studio Express (also installed OK) appears to connect to the database server and shows the database tree as I would expect, consisting of Databases, Security, Server Objects, Replication, and Management.

However, when I attempt to create a new data source (Control Panel->Administrative Tools->Data Sources(ODBC ->ODBC Data Source Administrator -> Add->SQL Server)

Name: MYSQLS1

Descr: Test SQL Server

Server: (local)

(Next->)

x With Windows NT Authentication using the network login ID

[Client Configuration]

Server Alias: MYSQLS1

Server Name: MYSQLS1

x TCP/TP

x Dynamically determine port [1433]

(Next->)

RESULT:

Connection failed:
SQLState: '01000'
SQL Server Error: 11001
[Microsoft][ODC SQL Server Driver][TCP/IP Sockets]ConnectionOpen(Connect()).
Connection failed:
SQLState: '08001'
SQL Server Error: 6
[Microsoft][ODC SQL Server Driver][Shared Memory]Specified SQL Server not found.

Can anyone please tell me what's going on here ? What am I doing wrong?

View 15 Replies View Related

Integration Services :: Source Script Component To Read Data From Sharepoint List?

Apr 29, 2015

All examples I found refer to classes under Microsoft.SharePoint namespace. However, I have the SharePoint CSOM that only gives me the Microsoft.Sharepoint.Client namespace.

I need to read the selected values of a multichoice field, but not sure how to do it with classes in the namespace above.

everthing works, exept the TSQL_x0020_Reference_x0020_Numbe field.

my code looks like this:

Webweb = cont.Web;
cont.Load(web);
cont.ExecuteQuery();
Listsstest = web.Lists.GetByTitle("T-SQL
Code Review Tracking");
//CamlQuery query = CamlQuery.CreateAllItemsQuery();

[code]....

View 2 Replies View Related

How Large Can XML Data In A Variable Be For The XML Source?

Aug 16, 2007

We can pass XML to the XML Source in a variable, but I haven't seen anywhere how much data can be passed this way? Is there a limit beyond the limits of system memory?

Also, what data types are valid for the variable? Just String?

View 4 Replies View Related

Using Object Variable As Data Source?

Nov 17, 2006

Hi!

These problem does't look like a problem, but i am not able to solve it...

I have a stored procedure (in an Execute SQL-Task) what returns a few rows. These rows a written in an object variable. No problem.

Now i want to use these result set in a dataflow as source. How can i manage these?

1. Use a script task what returns the whole result set from the object variable? How?

2. Is there any other possibility to use the results of a stored procedure in the data flow?

All ideas are urgently welcome...

Torsten

View 6 Replies View Related

Unable To Generate Model Off Of An Analysis Services Data Source

Feb 21, 2007

Hello,

Our company is doing some basic research into using analysis services as a solution for our customers. I've taken our database and built an analysis services project in BI Studio and deployed it to Analysis Services on my local machine I connect to Reporting Services in Management Studio, create a folder for the cube, give my Windows user acct. Content Manager permission, create the Data Source (conn string = "Provider=SQLNCLI.1;Data Source=(local)SQLServer2k5;Integrated Security=SSPI;Initial Catalog=STI-PDB" -- copied from the Analysis Services project in BI Studio), and go to "Generate Model". What I get is an rsModelGenerationError, sub [Create Perspective From Cubes] with a connection issue at the base:

Either the user, <computername>Thomas, does not have access to the STI-PDB Database, or the database does not exist. () (Microsoft SQL Server 2005 Analysis Services)

Now, in the Database Engine, my Windows account has the db_owner database role for STI-PDB and it's default schema is dbo, which all the objects are created under. What am I missing here?

View 4 Replies View Related

Urgent: How To Get Source Data With Runtime Variable

Feb 15, 2006

All,
I need to pass a variable to get source data from Oracle on OLE DB source. The variable is defined by an Execute SQL task (previous step) on the control flow and it is a package level variable.
In the data flow, OLE DB source using €˜SQL command from variable€™. The variable sql command works when it just: Select *from table
But I need the variable SQL command like this:
Select * from table where last_modified_date > ? - - variable1
the variable1 is defined by anther Execute SQL task at runtime

Can someone give some light on this?

Many thanks

View 1 Replies View Related

Highly Variable Source Data...how To Handle It?

Apr 11, 2007

I'm working on the viability of using SSIS to process data files that are (currently) sent by email or FTP and processed by hand. They are .csv or Excel and have some columns that are required, some that are included but the data is discarded, usually have some anomaly (like columns out of place, or columns missing) from source to source, and if they have headers the actual headers are inconsistent from source to source (and sometimes from submission to sumission ). Some sources send only one file, others might send a dozen. Due to the nature of the sources - many are not very technically inclined and sometimes there are other factors involved in how the data is exported - there's not a lot I can do to leverage them into all using the exact same format. Creating an SSIS package for each source (over 100 in all) is not a viable solution.

The compromise that I've come up with is I would like to come up with a universal SSIS package that can:


Take a .csv file with headers (I think we can force the sources to include the right headers if nothing else) as input , with or without quote delimiters.
Not be sensitive to the order of the columns.
Not be sensitive to the number of columns.
Loop through multiple files from one source.
Place the ouput in a staging table for further processing.
Reading a .csv file, looping and the staging table is not a problem. What I'm struggling with is the middle portion. I've tried to make the column headers dynamic, but then I lose the ability to have quote delimiters. My next instinct was to the source file(s) and create one interim file with a different delimiter (such as pipe or ## or something) but that doesn't seem to work quite the way I want it - since the source has comma delimiters, the output has both the pipe and the comma. Another way I could do it is replace the commas within quote delimiters with a blank space in the data flow, which would eliminate the quote delimiters in the output. That I haven't tried yet.

Any suggestions? If I could bring down the hammer and tell every source "You must send it this way" I would, but my hands are tied.

View 4 Replies View Related

Connection Manager Not Showing New Connection From Data Source...

Mar 30, 2007

Hello,

I've created a SSIS Solution and have created Data Sources. I have two packages. One was created before the Data Sources, and one was created after. The package that was created after is using connections from the Data Sources. I want to change the package before the Data Soruces were created to use them, but when I right click in the Connection Managers pane "New Connection From Data Source.." is not an option.

Did I not add it to the Solution properly?

How do I get it to show?

Did I not refresh something?

Please provide the how if you figure it out.

Thanks

View 4 Replies View Related

Unable To Retrieve Data From A Source CSV File That Contains Embedded Text Qualifiers

Apr 2, 2007



Hi Mates,

I am unable to fetch data from Source which is CSV file whose sample contents are given below:

-----------------------------------------------------------------------------------------------------------

"Fiscal year";"Posting period";"Company Code";"Functional area";"Profit Center";"Business area";"Group Account";"Posting Level";"Document Type";"Accounting Principle";"Cost Pool";"Financial Owner";"Period Value GC"
"2006";"12";"AR00";"A107";"P1131";"7200";"3410";"10";"18";"GAAP";"#";"#";"$ 0.00"
"2006";"12";"AR00";"A107";"P1131";"9600";"3410";"10";"18";"GAAP";"#";"#";"$ 0.00"
--------------------------------------------------------------------------------------------------------------



The error message that comes while I preview the source data:

Error:

"The Preview sample contains embedded text qualifiers. The flat file parser doesnot support embedding text qualifiers in data. Parsing columns that contain data with text qualifiers will fail at runtime"



Is there any alternative to remove these text qualifiers from the file. Do we have any utility that can convert these files into text qualifier free CSV file?

View 3 Replies View Related







Copyrights 2005-15 www.BigResource.com, All rights reserved