MS SQL Server :: SQL 2012 :: How To Make Sure That All Rows Inserted Into Datamodel And Then Truncate Staging Table

SQL Server 2012 :: Data Transfer From Staging Table Into Main Table

Jan 14, 2014

What is the best way to transfer data from the staging table into the main table.

Example:
Staging Table Name: TableA_satge (# of rows - millions)
Main Table Name: TableA_main (# of rows - billions)

Note: Staging table may have some data same as the main table.

Currently I am doing:
- Load data into staging table (TableA_stage)
- Remove any duplication of rows from the staging table (TableA_stage)
- Disable all indexes on main table (TableA_main)
- Insert into main table (TableA_main) from staging table (TableA_stage)
- Remove any duplication of rows from the main table using CTE (TableA_main)
- Rebuild indexes on main_table (TableA_main)

The problem with the above method is that, it takes a lot of time and log file size grows very big.

View 9 Replies View Related

SQL 2012 :: Load Consolidated Members Into MDS Staging Table - Hierarchy Name

May 28, 2015

I'm attempting to load some data into an explicit hierarchy in MDS 2012 via the staging table and struggling with the HierarchyName field. Specifically I'm loading data into stg.[Entity Name]_Consolidated and using the exact name of the explicit hierarchy I've set up in the front end web application.

Originally my hierarchy was labelled "Reporting Hierarchy" and when loading the data into staging using this name then running the batch from the Import Data screen I can see the error message "Error - The HierarchyName is missing or is not valid.". I've checked the table mdm.tblHierarchy and can see that the name there is exactly as it was in the staging table and have since renamed the hierarchy as "Reporting_Hierarchy" with the same results.

View 0 Replies View Related

SQL Server 2012 :: How To Create Staging Table To Handle Incremental Load

Jan 2, 2014

We are designing a Staging layer to handle incremental load. I want to start with a simple scenario to design the staging.

In the source database There are two tables ex, tbl_Department, tbl_Employee. Both this table is loading a single table at destination database ex, tbl_EmployeRecord.

The query which is loading tbl_EmployeRecord is, SELECT EMPID,EMPNAME,DEPTNAME FROM tbl_Department D INNER JOIN tbl_Employee E ON D.DEPARTMENTID=E.DEPARTMENTID.

Now, we need to identify incremental load in tbl_Department, tbl_Employee and store it in staging and load only the incremental load to the destination.

The columns of the tables are,

tbl_Department : DEPARTMENTID,DEPTNAME

tbl_Employee : EMPID,EMPNAME,DEPARTMENTID

tbl_EmployeRecord : EMPID,EMPNAME,DEPTNAME

How to design the staging for this to handle Insert, Update and Delete.

View 9 Replies View Related

SQL 2012 :: Using Excel In SSIS To Import Data From Spreadsheet To Staging Table?

Feb 5, 2015

I'm trying to use Excel in SSIS to import the data from spreadsheet to a staging table. The package runs well from the web server using SSMS. But when I deploy and try to execute the package, I'm getting the below error. I've a question, whether I've to install the AccessDatabaseEngine driver in SQL database server or the web server where I'm executing the SSIS?

Error: The requested OLE DB provider Microsoft.Jet.OLEDB.4.0 is not registered. If the 64-bit driver is not installed, run the package in 32-bit mode.

View 3 Replies View Related

SQL Server 2012 :: Trigger Inserted Multiple Rows After Insert

Aug 6, 2014

I create a Trigger that allows to create news row on other table.

ALTER TRIGGER [dbo].[TI_Creation_Contact_dansSLX]
ON [dbo].[_IMPORT_FILES_CONTACTS]
AFTER INSERT
AS

[code]...

But if I create an INSERT with 50 rows.. My table CONTACT and ADDRESS possess just one line.I try to create a Cursor.. but I had 50 lines with an AdressID and a ContactID differently, but an Account and an AccountId egual on my CONTACT table :

C001 - AD001 - AC001 - ACCOUNT 001
C002 - AD002 - AC001 - ACCOUNT 001
C003 - AD003 - AC001 - ACCOUNT 001
C004 - AD004 - AC001 - ACCOUNT 001
C005 - AD005 - AC001 - ACCOUNT 001

I search a means to have 50 lines differently on my CONTACT table.

C001 - AD001 - AC001 - ACCOUNT 001
C002 - AD002 - AC002 - ACCOUNT 002
C003 - AD003 - AC003 - ACCOUNT 003
C004 - AD004 - AC004 - ACCOUNT 004
C005 - AD005 - AC005 - ACCOUNT 005

View 9 Replies View Related

SQL Server 2012 :: Number Of Column Mismatch In Source File And Staging Table

Apr 14, 2015

I am having one store procedure which use to load data from flat file to staging table dynamically. everything is working fine.Staging_temp table have single column.All the data stored in that single column below is the sample Data.

AB¯ALBERTA ¯93¯AI
AI¯ALBERTA INDIRECT ¯94¯AI
AL¯ALABAMA ¯30¯

[Code]....

View 2 Replies View Related

SQL 2012 :: Single Column In Staging Table Deems Retrieval Query Unresponsive

Jul 17, 2015

where including/excluding a single column in an empty staging table would influence a resultset returning from distributed query? Both servers are SQL Server 2012. Nothing special about the staging table. It contains 12 columns with a mixture of INT and NVARCHAR(256) columns. In one case I exclude the column and the query returns in 17 seconds. When I include it the query does not return. Excluding the INSERT INTO the staging table and query returns in 17 secs with and without the column.

View 3 Replies View Related

SQL 2012 :: Group Concat To Make 2 Rows Into 1?

Oct 14, 2014

I have 2 different types of data for the same key. My queries renders these results into 2 rows. I'd like them to be one row.

Example

DOG Name_a Color_a Age_a Name_B Color_B Age_B
Boxer BO Fawn 1
Boxer Maggie Brindle 4
Lab Jackson Yellow 2
Lab Sandie Black 3

The result I want

DOG Name_a Color_a Age_a Name_B Color_B Age_B
Boxer BO Fawn 1 Maggie Brindle 4
Lab Jackson Yellow 2 Sandie Black 3

View 1 Replies View Related

Is It Ever Possible That 2 Different Rows Are Inserted At Same Time Into A Table?

Jul 27, 2007

There is a stored procedure that inserts a row into 'Vendors' table. Is it possible that two different calls to this sp happen at the same time and as a result, each sp inserts into the table its row at exactly the same time?

View 1 Replies View Related

Truncate A Huge Table Having 21 Millions Of Rows

Nov 19, 2015

SQL Server: 2008 R2

Question A : I need to truncate a table, it has 21 millions of rows and it has a size of 14 GB.

1- How do I find out if this table is not being referenced by a FOREIGN KEY?
2- Does it Participates in a indexed view?
3- Is being published by using transactional replication or merge replication?

Question B: How do I safely truncate that table?

View 8 Replies View Related

Rows Not Being Inserted Regularly Into Database Table

Mar 28, 2007

Hello All,

Doe anyone know if any limitations with SQL Server Express may be causing this?

I have an application (a web service hosted on my local PC) that parses an InfoPath form and submits the data to a database (SQL Server Express). The problem I'm have is that new records are not always inserted into the database when I submit. There is an autoincrement ID field and I observed that every other record was successfully inserted (a day ago) and today I observed that the records were inserted intermittently. However, when I run the same application on a laptop, the records are inserted each time.

I posted the associated code on the Data Access forum but it occurred to me that it may be because of the nature of SQL Server Express.Any suggestions would be much appreciated. Thanks!

View 1 Replies View Related

Get The Id Of Inserted Rows --&&> Insert Into Table1 Select * From Table 2 ?

Apr 17, 2008

hi

I want to do a "bulk" insert and then get the id (primary key) of the inseret rows:

insert into table1 select * from table2

If I get the value of the @@identity, I always get the last inserted record.

Any idea how to get the ids of all inserted values?

Thx

Olivier

View 13 Replies View Related

SQL 2012 :: Replication Fails - Cannot Truncate Table

Apr 24, 2015

We have an issue where we are trying to replicate a DB using SQL replication, the Replication Monitor reports an error:

Cannot TRUNCATE TABLE 'dbo.xxx' because it is being referenced by object 'View_xxx'.

Although I am the DBA, I do not have an inside knowledge of this particular DB.

View 2 Replies View Related

Transact SQL :: Truncate And Insert Operation - Avoid No Rows In Table During Processing

Oct 22, 2015

I have a job which executes hourly.

Essentially:

Begin
Truncate table A
Insert into A
(Col1,
Col2,
Col3...
)
Select Value1,
Value2,
Value3...
From Table B
End

The insert operation query takes approximately 3.5 minutes to execute. What's occurring is the Table is immediately truncated, and there are no rows in the table for those 3.5 minutes.

How can I avoid having this gap - where there are no rows in the table for that period of time during the job execution ? The table could be locked, but that doesn't seem like the best solution.

View 8 Replies View Related

SQL 2012 :: Alert Not Being Inserted To Sysalerts Table

Jul 17, 2015

I've built 4 new SQL2012 Ent SP2 instances using our standard build docsscripts all of which include an alert to pick up Sev 14 bad login messages.

However when I force a bad login to test these, they work on 2 servers but not the other 2.

The ones that fail write the Sev 14 msg to the SQL Error log and Windows Eventvwr fine, but there is no entry in the sysalerts table!?.

The alert is enabled and the windows event log service is running. I've noticed that none of the other alerts I've set up write to the sysalerts table though?

i'm not concerned about that yet, just the fact its not writing to the sysalerts table in the first place.

View 1 Replies View Related

Transact SQL :: How To Delete Newly Inserted Rows Automatically From Table After 30 Minutes

Sep 29, 2015

I would like to perform autodeletion operation from my table using stored procedure....

Here is my table structure:

Emp.id | emp_name | dept | desig | time_out | date | emp_sign |

if the user inserts new employee record that record should delete automatically from original after

30 minutes..I don't want to keep that newly inserted record after
30 minutes...I don't know how to achieve this...

View 12 Replies View Related

Inserted Rows Count From SSIS Not Like Table Rows Count

Jun 25, 2007

Hi all

i using lookup error output to insert rows into table

Rows count rows has been inserted on the table 59,123,019 mill

table rows count 6,878,110 mill ............................

any ideas

View 6 Replies View Related

SQL 2012 :: Check Command That Ensures Only Certain Values Inserted In Table?

Jan 14, 2015

I am trying to create a check command that ensures only A'B','c','D','E','F','G and s1, s2 can be inserted in the table, is this even applicable? Heres my code:

Create Table GRADE (
GradeVARCHAR2(1) CONSTRAINT pk_Grade PRIMARY KEY
CONSTRAINT check_grade
CHECK (substr(Grade = 'A','B','c','D','E','F','G')),

Salary_Scale VARCHAR2(2) CONSTRAINT check_SScale
CHECK (substr(Salary_Scale = 'S1', 'S2')),
)
/

View 1 Replies View Related

SQL Server 2012 :: How To Get A Table Identity Inserted By Instead Of Insert Trigger

May 14, 2015

I have a problem described as follows: I have a table with one instead of insert trigger:

create table TMessage (ID int identity(1,1), dscp varchar(50))
GO
Alter trigger tr_tmessage on tmessage
instead of insert
as
--Set NoCount On
insert into tmessage

[code]....

When I execute P1 it returns 0 for Id field of @T1.

How can I get the Identity in this case?

PS: I can not use Ident_Current or @@identity as the table insertion hit is very high and can be done concurrently.Also there are some more insertion into different tables in the trigger code, so can not use @@identity either.

View 5 Replies View Related

SQL Server 2012 :: In Trigger - Building Dynamic Table With Inserted Data

Nov 4, 2015

Within a trigger, I'm trying to create a unique table name (using the NEWID()) which I can store the data that is found in the inserted and deleted tables.

Declare @NewID varchar(50) = Replace(convert(Varchar(50),NEWID()),'-','')
Declare @SQLStr varchar(8000)

Set @SQLStr= 'Select * into [TMPIns' + @newID + '] from inserted'
Exec (@SQLStr)

I get the following error: Invalid object name 'inserted'

I know I can do:

Select * into #inserted from inserted
Set @SQLStr= 'Select * into [TMPIns' + @newID + '] from #inserted'
Exec (@SQLStr)

But I don't want to use TempDB as these tables can become big and I also feel that it is redundant. Is there a way to avoid the creation of #inserted?

View 2 Replies View Related

SQL 2012 :: Cannot Truncate Table Because It Is Published For Replication Or Enabled For Change Data Capture

Jul 8, 2015

I get the following error message when a job calls a Stored Procedure that TRUNCATES a Table:

Cannot truncate table 'CombinedSurveyData' because it is published for replication or enabled for Change Data Capture

Is my only option to change the TRUNCATE to DELETE?

[URL]

View 2 Replies View Related

Make A Bunch Of Rows From A Table Resemble A Column

Nov 23, 2005

Currently I'm using a UDF and a physical temp table to accomplish thisand I want to know if there's any way of doing it in a query orsubquery instead...In TableA there are a bunch of rows:InvitationID (PK) PartyID Partygoer1 1 Jim2 20 Bob3 2 Frank4 1 Robert5 1 PeteIn TableB are rows:PartyID (PK) PartyDate PartyName1 1/1/2000 WildParty2 1/1/2000 BoringPartyWhen a user runs a query to search for all parties on 1/1/2000 I wantthe result to look like:PartyID PartyDate PartyName Partygoer1 1/1/2000 WildParty Jim, Robert, Pete2 1/1/2000 BoringParty FrankI'm hoping there's a solution to this.Thanks,lq

View 9 Replies View Related

How To Make A Table To Display Perticular Number Of Rows Per Page??

Nov 29, 2007

Hi,
Can anybody tell how to ristrict the number of rows per page in SSRS?
I have a table in my report.I know we have to use" =Int(RowNumber(Nothing)/25)" in group expression.But I already have 3 group expression in my table.Where to write??
Thanks.

View 9 Replies View Related

How To Make Sure In An Insert Statement That The Vendor Being Inserted Does Not Exist?

Dec 3, 2006

I am using INSERT into Vendors (.....) VALUES (....) form of insert statement. How can I make sure that the insert fails if thie new vendor's vendorname exists?
I cannot create a unique index on 'VendorName' column since it is more than 900 bytes and SQL Server will not allow to create index on a column bigger than 900 bytes.

View 8 Replies View Related

SSIS Loading DWH Staging Area When Table Names Is Selected From Table List

Aug 31, 2007

Hello,
Maybe anyone have done that before?
I have table where i store SOURCE_TABLE_NAME and DESTINATION_TABLE_NAME, there is about 120+ tables.
i need make SSIS package which selects SOURCE_TABLE_NAME from source ole db, and loads it to DESTINATION_TABLE_NAME in destination ole db.

I made such SSIS package. set ole db source data access mode to table or view name variable.
set ole db destination data access mode to table or view name variable. set to variables defoult values (names of existing tables)
but when i loop table names is changed, it reports error, that can map columns, becouse in new tables is different columns.

how to solve that problem?

View 5 Replies View Related

SQL Server 2008 :: Inserting Data From Staging Table To Main Table

Feb 23, 2015

I am trying to insert bulk data into main table from staging table in sql server 2012. If any error comes, this total activity is rollbacked. I don't want that to happen. I want to know the records where ever the problem persists, and the rest has to be inserted.

View 2 Replies View Related

Merging Master And Staging Table (was: Update Table Using SSIS)

Nov 20, 2007

Hi,
Is this anyway to finding updated/ deleted recored using anyother data flow transformation tasks without using sql task.
Can find the new records using merge join task.

Is there better way to merge master table using staging table?

Thanks in advance.

DF.

View 1 Replies View Related

DB Design :: Insert / Update FACT Table From Staging Table

May 6, 2015

We need to Insert/Update a Fact Table from staging Table. currently we are using a SP which update Fact Table for Each region. this process is schedule, every 5 min job is run and Update fact table.but time of Insert and Update too long from staging to Fact, currently we are using merge statement for Insert and update.in my sp we are looping number how many region we need to update and at a time single Region we are updating using while loop in current SP.

View 7 Replies View Related

What Technique To Use To Update Production Table From Staging Table

Mar 9, 2007

Hi I have a production table in SQL Server 2005 that has approx 500,000 records---every 6 hours this table needs to be truncated and filled

The basic SSIS package uses a script compentant and imports the data into a staging table which has the same structure as the production table. I have a final Execute SQL Task that Truncates the production table and does a Insert into production-select * from stage table.

Takes around 30 seconds to run this last Execute SQL Task--problem is there is a risk that our webservice will query this table during the Execute SQL Task and return incorrect results.

Q1: In this last Execute SQL Task if I used a BEGIN TRANSACTION and COMMIT TRANSACTION; will this be any quicker ?

Q2: In this last Execute SQL Task- would it be better to use a RENAME TABLE technique in TSQL--any code examples ??

Q3:Is there any way in TSQL I can compare EVERY FIELD in two tables ie Stage and Production which have identical data structures and figure out a way to update only the records that changed? Is SQL Server Replication the best way to do this

thanks kindly

Dave

View 9 Replies View Related

SQL Server 2012 :: Create Table From Rows Of A Different Table

Mar 6, 2015

View 9 Replies View Related

CSV To Staging Table Import Using BCP

Mar 31, 2008

Hi Guys,

I'm trying to import the contents of a CSV file into a staging table that I've created in SQL server 2005. To perform the import I have used the BCP utility with the use of a BCP format file.

The problem I'm having is that the data in some of the fields in the csv file are longer than the length of the corresponding field in my staging table. So when I try to import I get the following error:

[Microsoft][ODBC SQL Server Driver]String data, right truncation

And the record with the error does not import.

My question would be, is there a way of telling BCP to trim the
data before importing into the staging table? Could I somehow
specify in the BCP file to trim the data before importing or
is there a switch that i can specify in the BCP command which tells BCP to trim data to the length of the destination column.

If it helps I'm using the command below to run BCP.

bcp S_OLTPDEV.dbo.CX_LOTS_STG in %path1%CTS_SiebelLotsTo_Siebel_From_CTSlotupdates.csv -f %path1%CTS_SiebelLotsTo_Siebel_From_CTSLots_Format.fmt -F 1 -S %dbsrvr% -U %duname% -P %dpasswd%

Thanks in advance, any help would be really appreciated.

View 3 Replies View Related

Populating Staging Table Using SSIS

Oct 6, 2006

Hi,

I have a flat file with columns from a geographical hierarchy such as:

Country Zone State County City Store Sub Store , etc.

The file also has data columns for months to the right of the above columns such as:

Jul Aug Sept ......... basically 25 of these columns for two years' data for one product and another set of 25 columns for another kind of product. A typical record in the file looks like:

Country Zone State County City Store Substore

USA Southeast FL Hillsborough Tampa walmart Fletcher

May04_ProdA ....Jun06_prodA May04_ProdB...Jun06ProdB
144 160 180 158

I need to upload this data into a staging table in SQL Server 2005 using SSIS, I created a table with the geographical hierarchy columns but am trying to figure out a way to load the monthly data. I can create 50 columns for the 50 months ( 25 months for each product) but that would be very crude.

Is there a better way of inserting data from this flat file into a destination table? I need all the data in the staging table in one upload.

Thanks.

View 11 Replies View Related