I know parsing json data has been discussed lots but what I want is probably a little simpler or different:
I have a URL where I can open and get the Json data.
I need to parse and load the Json data into a SQL table, and I want to use it in SSIS, not using C# or VB.NET coding.
I am on Visual Studio.NET 2008 and SQL 2008R2
I have a below JSON script. I need to run the below script, fetch the json files and parse them in SSIS.
var client = new Keen({
projectId: "<PROJECT_ID>",
readKey: "<READ_KEY>"
[code]....
I tried using script component, but unable to find how to capture these JSON.
While importing data from Excel source , some column is getting null value even though excel column has value.To Resolve the issue we tried with
HKEY_LOCAL_MACHINESOFTWAREWow6432NodeMicrosoftOffice14.0Access Connectivity EngineEnginesExcel
1.Change the Value of the Row TypeGuessRows from 8 (Default value) to 0 and ImportMixedType = text
• xls
HKEY_LOCAL_MACHINESOFTWAREMicrosoftJet4.0EnginesExcel
1.Change the Value of the Row TypeGuessRows from 8 (Default value) to 0 and ImportMixedType = text
the connection string of the excel
UPPER(REVERSE(SUBSTRING( REVERSE(@[User::VarInputExcelFile]), 1, 5) ) ) == ".XLSX" ? "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" + @[User::VarInputExcelFile] + ";Extended Properties="Excel 12.0;HDR=Yes;IMEX=1";":"Provider=Microsoft.Jet.OLEDB.4.0;Data
Source=" + @[User::VarInputExcelFile] + ";Extended Properties="EXCEL 8.0;HDR=Yes;IMEX=1";"
by doing the above setting also , the column is coming as null from excel source even though there is data in excel.
I am trying to import an xlsx spreadsheet into a sql 2008 r2 database using the SSMS Import Wizard. When pointed to the spreadsheet ("choose a data source") the Import Wizard returns this error:
"The operation could not be completed" The Microsoft ACE.OLEDB.12.0 provider is not registered on the local machine (System.Data)
How can I address that issue? (e.g. Where is this provider and how do I install it?)
I have list (in an input file) where each row is about 20K in size (so it can't be stored in a sql table). I want to convert the list into a table as shown below:
before:
---------------
pk1, c1, d1, c2, d2, c3, d3,......
pk2 c1,d1,c2,d2,c3,d3.....
where "pk" denotes a primary key and in a given row (c1,c2.c3,...) are all distinct. ("c" are columns names, "d" is the associated data)
after: (desired conversion)
---------------
pk1,c1,d1
pk1,c2,d2
pk1,c3,d3
...
pk2,c1,d1
pk2,c2,d2
pk2,c3,d3
....
I was planning to have SSIS pull in the "before" data, run a custom C# program in SSIS against it to massage the data to vertical (3-column format), then export the massaged data to a new text file. The new text file would later be imported into a sql table using SSIS.
I created a simple SSIS package that takes a Flat File Source (CSV file) and Imports it into a OLE DB Destination ([TestCSVImport].dbo.Table1). I have other CSV files I'd like to import, but I don't want to import entries where column "ordereID" (PK) are the equal. Just want to import the new data found in the CSV files. I tried adding a Lookup in-between the Flat File Source and the OLE DB Destination, but I'm not sure how to accomplish only importing new data.
View 2 Replies View RelatedThe only way to add a new column to an existing mapping that I know is to go to advanced editor and refresh. This however keeps only the default mapping (where the field names match), the rest is wiped out, so need to restore the mapping manually after that. Risky and annoying at the same time. Is there any alternative?
View 3 Replies View RelatedHi,
I've a xml file which I need to send over http to another server. The other server processes xml file and sends back xml response over http.
How can this be achieved in SSIS.
Thanks
Manish
Hi,
I've a xml file which I need to send over http to another server. The other server processes xml file and sends back xml response over http.
How can this be achieved in SSIS.
Thanks
Manish
Hi,
I have deployed the reports to the report server(centralized server). when i try to access the reports directly thru report server i.e http://servername//ReportServer in my local system. it asks for the username/password, when i give the username and password it accepts and shows me the report.
the same report i try to access in .NET Application, i do not know how to pass the credentials and it throws the error :"The request failed with HTTP status 401: Unauthorized ".
I am using VS 2005 and SQL Server 2005 and SQL Reporting Services 2005.
with regards
Suresh Babu
We are using SharePoint 2010 integrated with Reporting Services 2008 R2. We have successfully configured RS and we are able to create RDL reports using Report Builder from the SharePoint site.
However we are getting following error message when we host same RDL report using Report Viewer control in an application page i.e. ASPX page:
<ERROR>The request failed with HTTP status 401: Unauthorized</ERROR>
We can fix this issue by setting:
1) setting impersonation to false in web.config file:
<identity impersonate="false" />
2) Setting Report server URL to _vti_bin path:
rptViewer.ServerReport.ReportServerUrl = new Uri("http://sharepointserver:1000/_vti_bin/reportserver");
If we turn impersonation to true we get the same error. Other project module requires impersonation to be enabled. Hence we cannot proceed further with this work around/fix.
How to get data from SAP using SSIS.
View 3 Replies View RelatedIs it possible to deploy or import an SSIS package developed in visual studio 2010 onto a 2008 R2 integration server?
package: developed in vs2010, SQL Server: SQL Server 2008 R2.
Trying to deploy the package onto the server.
I get this error message when I try to connect to Reporting Services via the Management Studio.
I can see my machine listed in the Server Name > Browse For More > Local Servers dialogue. But no luck,
Ive tried:
Servername: localhost
Servername: DED1774 (the machine name)
Servername: localhost/reportserver
Servername: DED1774/reportserver
Servername: http://ded1774/reportserver (from the rsreportserver.config file
<UrlRoot>http://ded1774/reportserver</UrlRoot>)
I've Googled the error message and found postings for solutions, but none of these helped. Can anyone suggest some simple steps I can take to try to find the issue and get the connection working?
Thanks
I'm running SSIS 2008 (VS 2008) on a windows server 2008 R2 server. I added a new ODBC driver (Specifically for SAP HANA) and then configured and successfully tested an ODBC System dsn in the 64 bit odbcad.exe.
Unfortunately when I go to use this system dsn in SSIS 2008, the "Configure ODBC Connection Manager" is empty. After some research it appears that by default SSIS 2008 (or at least my particular instance for whatever reason) brings up the list of installed ODBC drivers/User and System dsn for 32 bit (C:WindowsSysWOW64odbcad.exe) instead of the 64 bit connections (C:WindowsSystem32odbcad.exe). I'm wondering if there is a setting somewhere where i can tell SSIS to use the 64bit connections by default in lieu of automatically going into C:WindowsSysWOW64odbcad.exe or if there is some other work around here ?
At one of my accounts I'm using SQL 2008r2 and SSIS in an DWH solution. Today I wanted to created a dynamic sql statement in one of the packages. The dynamic part is in the where-clause a sql query I use. I want to set the values for this clause dynamically. Therefor I created a user variable (named 'Filter') which should get it´s value at run-time.
/DTS "MSDBProdMy Package" /SERVER "My-Server" /CHECKPOINTING OFF /REPORTING EW /SET "Package.Variables[User::Filter].Properties[Expression]";" where [Starting Date] < '2008-09-01 00:00:00.000' "
However when I execute this package with dtexecui it doesn not use the value specified after the /SET parameter, but instead uses the value specifiied at design time. I get the same when I add the execution of ths pacakage as a sql server job step (type SSIS). I would expect that value specified after the /SET parameter would overwrite the valus specified at design time, but I don't see that happening.
I want to import a data file into a sql table. The table has a primary key but the data could have a duplicate value in the PK column (error in the source data). How can I "trap" for this type of error in SSIS?
View 10 Replies View RelatedSSIS 2008 when I develop and debug in BIDS sometimes ignores debug break point.
The script component is in the main control flow and at some point the breakpoint did work.
If, for example, I create a new project and copy my script component there the debug breakpoint will work.So it's absolutely *random* when it works and when it does not.
Below is my BIDS detail:
Microsoft Visual Studio 2008
Version 9.0.30729.4462 QFE
Microsoft .NET Framework
Version 3.5 SP1
Installed Edition: IDE Standard
Enterprise Library v5 Configuration Editor 4.0
[code]....
I have a flat file which have some record data ex.
id name team
1 "A"my" "Bl"ue"s"
2 "Bob" "Reds"
3 "Chuck" "Blues"
4 "Dick" "Blues"
in above example first record contain invalid data so complete flat file will not import due to one invalid row or record in flat file. so is there any way to check invalid row from flat file and ignore it(write log about invalid record) and process importing flat file.
I have a flat with few columns
FirstName, lastName, Address
f1,l1,a1
f2,l2,a2
I build my SSIS package based on the above file.But now i receive files with different columns order let say
lastName,FirstNamr,Address
l1,f1,a1
L2,f2,a2
or
Address,FirstName,LastName
a1,f1,l1
a2,f2,l2
every time i receive multiple files in different order and i have to remap all my mappings. These are just a few columns and i have like 20 columns and the order can potentially change any time. so every time i have build new packages remap them etc.
through normal c# code it pretty easy. I tried to add script here but the script also needs a source and mapping so there is also a mapping issue. Is there a better way to do this.
I have package which pulls data from db table and creates a excel file extract.The flow is like this - A excel file template sits in the input folder folder for processing .The package starts by dropping excel sheet in the excel(which is clearing any data and columns available) once that is done it has script task which creates a new columns for the sheet and gives a sheet name as well .Then a execute sql task runs and pumps data into a table which serves as a source for the excel extract process .The excel extract process involves pulling data from the table and doing data conversion before it moves it into the oledb destination (excel file on file server).When I run the package I go and see that data is pushed down . I see top rows say 100 are empty and data appears after say 100 rows .
I tried deleting excel file and replacing with new one empty with columns and sheet name only but still it doesnt work?I am trying to understand what is making ssis behave like this and what can I do overcome the problem ?I read on google that we need to bring in file system task will move a template to working directory which is input folder but I dont want it to incorporate that logic as we need to push this package to production ASAP with very minimal change.
How to split excel workbook into individual sheets using SSIS 2008 without using " Microsoft.Office.Interop.Excel"
The Workbook need to be broken into multiple sheets available in the workbook.
The SSIS package will read the excel workbook and split into individual sheet and rename the sheet with the name available in the workbook..
I'm trying to import some XLS files that I receive from some suppliers. The problem is that every time they send some columns with text values but formatted as number. When I read those columns with SSIS Excel Source, they come all with null values.
I don't want to change the columns data types every time, so I would like to know if there's a way to bypass the column types that are already there.
I tried to use both the Jet driver and the Office 12 driver. I've already used the IMEX=1 on ExtendedProperties too with no success. Is there a way to force reading the columns as text, even if they have data types assigned to them?
We built our prod server [vm] with SQL Server 2008 R2 on Server 2008 R2. It works nicely. Then we made a copy of the VM and renamed it (so our test environment would be IDENTICAL to production). After that, SSRS was broken: I get "HTTP Error 503. The service is unavailable." I can't connect via http, or SSMS.
We have tried:
* Running SSRS config tool (several times)
* Running with/without the IIS Server Role
* Dropping & recreating the SSRS keys
* Recreating the ReportServer database, etc
* Checking all of the accounts, permissions, etc
* Running SQL Repair
* Going through the registry to fix any references to the machine's old name
* Uninstalling, reinstalling SSRS
* Completely uninstalling ALL of the parts of SQL Server 2008 R2, deleting all directories, removing references to SQL Server from the registry, rebooting, reinstalling everything.
None of this has worked. SSRS [R2] is still 503 on our test box.
the SSRS [NT] service seems to run, without error. The Event Viewer doesn't seem to be recording any errors. The SSRS logs say that the default URL is wrong, but we get the same error in Prod, and Prod works fine. The other SQL Logs say something about not being able to contact the service. However, as I said, the [NT] service seems to be working fine.
I have multiple xml data file in a directory say C:XMLData abc1.xml, abc2.xml, abc3.xml etc.
Need to loop through each file in ssis with Foreach loop container, and get the file name say abc1, and load the data of abc1.xml to abc1 table in sql server DB.
Next iteration will pick up the abc2.xml and find the abc2 table in sql server DB then insert the data in abc2 table.
While each iteration, xml source should also point each xsd file correspondingly.
Tables are already created in DB
I solved my problem up to getting the file name from ech iteration and assigned file name to variable, in oledb destination data access mode I select Table or view name variable, then corresponding table will get selected for data insertation.
Just wanted to know how can I read each xsd file for each xml data files while iteration.
I have a simple SSIS package that reads a flat file and copies it into a SQL Server table.
When the flat fiel is on the C drive I have no problem runnign this package from SQL Server Agent, but as soon as I update the path to a network location the package only works when I run it manually, but fails when is executed via the SQL Server agent job.
The error says "cannot open the datafile", while the datafile location is valid.
Is this a kind of limitation of a SQL Server Agent that only local files are allowed to be processed?
import data form one table to another.Both table have different schema.Lets say
1. Employee(Empid, ename,address,designation,Joindate,DOB).
2.Person(name,address,DOB)
I need to import person table data into Employee table.
Note :
1 empid is auto increment
2. If Person.DOB is not present insert null into employee.DOB
3.JoinDate should be initialized with current date.
Currently I am using
1.OLE DB Source
2.Data Conversion
3.OLE DB Destination.
I am importing Differennt Excels Files into table. my require ment is after importing completed I need to insert all these Filenames ,File creation date into table. (for Auditing).
View 9 Replies View RelatedI have a requirement where in i have around 15 different flat files , filenames are fixed but folder path can be changed(i think i should use a variable for folder path). These 15 files data should go to their respective tables in the database.
Whether I need to create separate data flow task for each file or separate package? In addition to these, example : while importing product data into product table, if product ID already exists, we need to ignore it and upload only the new records.
I have an excel file that has multiple sheets and I need to import data from each separate sheet to a separate table using SSIS.
E.g. Sheet A data should go to Table A and Sheet B data should go to Table B and so on. Is it possible to do this with out using script task.
have a ASP.NET app that access the DBs in a SQL 2012 server. I added code to access the SSISDB but it returns an error " cannot open database SSISDB request by login"doing some basic troubleshooting:
-When login to SSM (2012-R2) with windows/sa credentials I can access SSISDB and its tables with no problem
-but when I try to access SSISDB via T-SQL as follows:
osql -Usa -Ppasswrod
>sp_helpdb...(this lists all the DBs including the SSISDB)
>use SSISDB
>go
>select * form TableName
>go
This script returns an error stating that SSIDB does not exist
I need to read in general web pages (not a web service) from a typical web site using SSIS and make it available for other SSIS transformations (Script Component). I tried using the XMLSource data source but this appears to require well formed XML, and will not accept HTML which is what I am likely to be getting from the web pages.
I tried a HTTP Connection Manager with a DataReader Source, but seems to only accomodate web services.
Can this be done? If someone has an example (tutorial) of how to accomplish this I would greatly appreciate a copy.
James
How can we create table partition and how can we use in ssis 2012.
View 6 Replies View Related