Wait For Incoming CSV Files
Jul 5, 2007are there any tasks in SSIS that watches a folder and waits for incoming files?
View 9 Replies
ADVERTISEMENT
Watching An Incoming Directory For Files To Process
Oct 21, 2005Here's the senario:
View 8 Replies View RelatedSQL 2012 :: TEMPDB High Avg Write Wait Time On Data Files
Apr 15, 2014I am currently investigating aa high avg write time ms issue (145ms) which seems to be only occuring on the tempdb data files.I have followed the recommended setup of TEMPDB in that
1. Data files = number of physical cores
2. Data files and logfiles are on separate partitions away from the other databases.
3. Tempdb is presized and no incremental file increases look like they are happening with frequency.
We have sharepoint 2012 setup on other sql servers and with TEMPDB setup following the same guidelines, with far more Sharepoint activity on a similary specified hardware which is why its confusing.FileIO auditing on the partitions themselves shows that the FileIO is very fast on the partitions that the tempdb data file which leads me to beleive that Sharepoint may be the culprit perhaps due to excess use of tempdb with operations taking a long time to resolve.
Processing Incoming Emails
Sep 14, 2004How would I go about checking incoming e-mails? For example, on a certain e-mail address, I would get e-mails formatted in a certain way. According to the response, some scripts need to run/ some sql tables updates etc. How can one do this in (ASP) .NET with SQL Server? Anyone did this kind of stuff before?
View 3 Replies View RelatedHow To Check For Incoming Date???
Mar 12, 2007I would like to check if the incoming date is valid date and i would also like to check if the date exists in my database.
i am transfering data from a flat file so all the data is string data.
How would i assign the datatype to the incoming columns as they exists in the sql table.Because it would better when i try to compare my incoming columns with the once in my database.
Here when i have a lookup transform and try to map one of my columns with string datatype to a column in my sql table of datatype date time i get a datatype mismatch error. How do i need to check if its date time...and how can i check if the incoming date is valid date.
please let me know
How To Remove From The Incoming Data
May 19, 2008Here is my sample data
Input text file:
"Name" "25" "00/00/00"
When i store the data in the database it get's stored with " " just the way it looks above
Expected output:I want to remove " " surrounding data.How do i remove???
eg: Name 25 00/00/00
Please let me know
thanks
How To Process Incoming EMAIL For SS2005?
Dec 20, 2007
ok, so sp_processmail is deprecated and should no longer be used in SS2005.
how do you process and execute a SS2005 query using an incoming email??
can it even be done in SS2005??
thanks for any ideas,
chester
Redirect Incoming Data To Different Tables On The Fly
Sep 4, 2007
Are there methods to redirect incoming data to different tables on the fly in the situation when application feeds data to the table via ODBC and I'm unable to alter ODBC or application settings?
Kind regards,
A.
How To Remove Duplicate Records From Incoming Textfiles
Apr 16, 2007Is there a way to check if duplicates exists in the incoming textfiles????
Capture System.IO Rows From Incoming File And Insert Into Table
Mar 24, 2006Ok, I'm not quite sure how to approach this one. This is a VB.NET console app in which I want to capture each row and throw it into a table. The reason being, they want a report on what was processed...which I'll be able to do easily in Reporting Services 2005 once this crap is in a table where it should be.
1) What should I use to do this, dataset? I want to use stored procedures also, not inline SQL
Function here takes an incoming file, and splits it up into separate files. I want to insert each row that is succesfully split
Public Sub ProcessFiles(ByVal sIncomingfile As String, ByVal sOutputDirectory As String)
If sIncomingfile <> "" And sOutputDirectory <> "" Then
Dim f As New Security.Permissions.FileIOPermission(Security.Permissions.PermissionState.None) f.AllLocalFiles = Security.Permissions.FileIOPermissionAccess.Read
Dim file As New IO.FileInfo(sIncomingfile) Dim filefs As IO.FileStream = Nothing If file.Exists Then Try filefs = New IO.FileStream(file.FullName, IO.FileMode.Open) 'Place: 1 Catch ex As Exception SendEmail("Incoming .mnt or .naf Filename Invalid or not found", "Place: 1") Application.Exit() End Try End If
Dim reader As New IO.StreamReader(filefs) Dim counter As Integer = 0
Dim CurrentFS As IO.FileStream Dim CurrentWriter As IO.StreamWriter Dim extension As String = IO.Path.GetExtension(file.FullName)
If extension = ".mnt" Then While Not reader.Peek < 0 Dim Line As String = reader.ReadLine If IsNumeric(Line.Substring(0, 1)) Then Dim Parts() As String = Line.Split(" "c) ' split row into parts If Parts(0).Length = 8 Then ' if first part is 8 then know we hit another header so cut and then write to file counter += 1 If Not CurrentWriter Is Nothing Then CurrentWriter.Flush() : CurrentWriter.Close() CurrentFS = New IO.FileStream(IO.Path.Combine(IO.Path.GetDirectoryName(sOutputDirectory), Line.Substring(59, 4) & "[" & counter.ToString & "]" & Now.ToString("MM-dd-yyyy") & IO.Path.GetExtension(file.FullName)), IO.FileMode.Create) CurrentWriter = New IO.StreamWriter(CurrentFS) End If
If Not CurrentWriter Is Nothing Then CurrentWriter.WriteLine(Line) End If
End If End While
If Not CurrentWriter Is Nothing Then CurrentWriter.Flush() : CurrentWriter.Close()
MoveFilesFTP(sOutputDirectory, "mnt")
ElseIf extension = ".naf" Then While Not reader.Peek < 0 Dim Line As String = reader.ReadLine If Not IsNumeric(Line.Substring(0, 1)) Then ' if first part is not a number, then we know it's a header so split the file counter += 1 If Not CurrentWriter Is Nothing Then CurrentWriter.Flush() : CurrentWriter.Close() CurrentFS = New IO.FileStream(IO.Path.Combine(IO.Path.GetDirectoryName(sOutputDirectory), Line.Substring(6, 4) & "[" & counter.ToString & "]" & Now.ToString("MM-dd-yyyy") & IO.Path.GetExtension(file.FullName)), IO.FileMode.Create) CurrentWriter = New IO.StreamWriter(CurrentFS) End If
If Not CurrentWriter Is Nothing Then CurrentWriter.WriteLine(Line) End If
End While
If Not CurrentWriter Is Nothing Then CurrentWriter.Flush() : CurrentWriter.Close()
MoveFilesFTP(sOutputDirectory, "naf") End If Else 'input file not valid SendEmail("Incoming .mnt or .naf Filename Invalid", "Place: 1") End If End Sub
Transact SQL :: Update Incoming Rows Based On Percentage Calculation
Apr 2, 2015This is on SQL Server 2008. Please find a detailed description and the file of the data, that I am working on.
Requirements:
1.
If
'Channel'
is
not
equal to "Omnibus"
where
the 'Trans Description'is
equal to "Purchase"
and
"Redemption"
for
one purchase and
one redemption that match on 'System'
,
'Account TA Number'
,
'Product Name'
,
'Settled Date'
,
and
where
the 'Trade Amount'
of the purchase and
redemption is
within 5%,
then
display those set of records.
2.
If
deemed wash trades,
allow user to update the purchase and
redemption pair 'Trans Description'
from
"Purchase"
to "Exchange In"
and
'Trans Description'
from
"Redemption"
with
"Exchange out"
System Channel Dealer Name Firm Name Product Cusip Product Name Product Share Class Trade ID Settled Date Account TA Number Trans Description  Trade AmountÂ
SCHWABPORTAL US - ASG MILLIMAN MILLIMAN 64128K777 Strategic Income Fund A 29806259 30-Jan-15 000BY00F2RW Redemption  $     25,68,458.15
[Code] .....
SQL Server 2008 :: Update Incoming Rows Based On Percentage Calculation
Mar 13, 2015I have below records coming in from source files
ProdName Amount TranType
P1 100 A
P1 100 S
P2 200 A
P2 205 S
In case the ProdName is same, and Amount = or (within +/- 5%) of Amount, I have to update the TranType column as IN/OUT respectively as shown below in the tables.
I am okay with using 2 different tables if needed as in the records comes in one table and then i can reference that table to upload the values in another.
ProdName Amount TranType
P1 100 IN
P1 100 OUT
P2 200 IN
P2 205 OUT
The order of the records coming in can be different order, they need not be subsequent.
This is happening in SQl Server 2008.
PAGEIOLATCH_SH Wait
Aug 1, 2000I am running into problems while running a large procedure, and i think it may have something to do with a PAGEIOLATCH_SH wait problem.
My server, whose sole purpose is to run this one procedure, is doing plenty of disk i/o, and the CPU’s bouncing around, so I assume it’s working. But when I look at its process info, it seems to be sleeping a lot of the time on PAGEIOLATCH_SH. No other users are in the DB, so I'm quite confused. I don't find much info on this anywhere, so any insight would be very appreciated.
Event Wait
Jan 15, 2008I'm new to SQL Sever 2005 and I'm trying to do what Informatica (Power Center - ETL) is trying does.
I have created a work flow and it is scheduled to run at every night 1:00 AM .The process is to load a flat file (CRV.data) into the database from a shared location.The flat file is transfered from a 3rd party and once the file transfer is complete it will create a indicator file (0 byte eg: CRV.DONE file) which indicates the CRV.data transfer is complete.
In my workflow I will be waiting for the CRV.DONE indicator file and once it is avaiable I will start loading the CRV.data and once the load is completed I will delete CRV.DONE file and be ready for the next day load.
Please let me know if there is any way in SQL Server 2005 to achieve it.Thanks
Wait For A Response?
Jun 11, 2007Hi. We are migrating a mainframe datacom database to SQL Server. One of our client-server applications already uses SQL Server. This application uses a middleware product to query and update the datacom database being migrated. We are considering using Service Broker to replace the middleware.
In many cases the client does not need a response provided the message is queued and will eventually get delivered. However, in some cases the client would like to wait for the message to be processed before proceeding. Is there an easy way to both submit and optionally wait for a response - with data - in a single stored procedure? If client does not want to continue to wait, is there a way to use a procedure to check for the returned message later?
We have not used Service Broker before and are doing for a "sanity" check before proceeding. We do not want to tightly couple the two databases at this time.
Wait Problem...
Dec 11, 2007I have installed performance dashboard on 2 different servers. The first server have User Session CPU Time 71% and Wait Time =28%, The other server have Cpu Time of 20% and Wait Time of 79%. Have I understand that stands in SQL Server Waits And Queues that I have some typ of wait problem in my second server?
Then I tries to run this
Select
'%signal waits' = cast(100.0 * sum(signal_wait_time_ms) / sum (wait_time_ms) as numeric(20,2)),
'%resource waits'= cast(100.0 * sum(wait_time_ms - signal_wait_time_ms) / sum (wait_time_ms) as numeric(20,2))
From sys.dm_os_wait_stats
First Server
%signal waits %resource waits
--------------------------------------- ----------------------------
0.07 99.93
Second Server
%signal waits %resource waits
--------------------------------------- ----------------------------
0.12 99.88
Messy in my head€¦ Help please
My second server have
CLR with 50% in the historial wait. Rest in Sleep
My first server have
99% in sleep wait category.
Sysprocesses Says: Wait Forever
Sep 30, 2004I'm doing an update on a table with about 113m rows, the update-statement is fairly simple: update tab set col = null where col is not null.
The col column is mostly null.
Sysprocesses shows three rows for this statement: 1 CXPACKET (its a dual processor, 2000 box with sp3 installed), 2 PAGEIOLATCH_SH (waitresource is filled). My guess would be that the where-clause is executed in a seperate process blocking the update.
I changed the statement into update [...] set col = null; sysprocesses shows one row with PAGEIOLATCH_SH. Executing forever.
I checked other processes including those outside sqlserver but none are using the db, let alone accessing the table involved. Even restarted sqlserver to be sure there's no dead process blocking the update. Didn't help.
So I added a search condition to the where-clause, involving a clustered index in order to reduce the rowcount. The execution plan shows a 97% hit on the clustered index, but sysprocesses shows the three rows again...
So far the profiler didn't help me out either: there's a SP: CacheInsert on the update-statement... then nothing.
What should I do?
Wait Type: NETWORKIO
Jan 14, 2005I have an ASP.NET web application that hangs on a single database UPDATE command for 5+ minutes. I can see this occur in SQL Profiler. This is a one row UPDATE statement on a small table (~600 rows). There are no JOINs or sub queries. There are no other users using the system. During this 5+ minutes, I can see the job in Enterprise Manager with a wait type of NETWORKIO. Since both IIS And SQL Server are running on the same system, the network shouldn't be an issue. Any ideas?
View 5 Replies View RelatedNETWORKIO - Wait Type
Sep 29, 2006Hello, my first post??
View 4 Replies View RelatedUsing BACKUP From OSQL And Wait For It's End
Aug 31, 2006I'm writing a small vbscript to backup a db and some related files, so I used a WSShell calling OSQL to run a Sql BACKUP command, then after it's finished I XCOPY the resulting file plus some other related files. But the problem is that OSQL ends it's execution as soon as the BACKUP command is sent to SqlServer, not when the backup itself ends.
Anyone knows how to synchronize the two? How to wait, inside OSLQ, for the end of the BACKUP execution?
TIA
Luigi
Causes For Latch Wait Time
Jan 6, 2008I setup a SQL Agent to send me an email when the Average Latch Wait Time is greater than 300ms. Now I receive an email every 15 seconds stating that the current ALWT is 3916ms. That value never changes with the emails. However, the perfmon shows nothing at all (shows zero).
I also have a Buffer cache hit ratio of 2848.00.
These numbers are when there is NOBODY on the DB at all It is just sitting there. When I reboot the server, as soon as SQL starts it starts to send the emails again.
Server: Intel Xeon Quad Core 2.66
RAM: 4GB (with /3GB in the boot.ini)
RAID 1: OS
RAID 1: Data (DB and logs)
CPU Utilization: 0-1%
RAM Utilization: 527MB
OS: Server 2003 R2 With SP2
SQL: 2005 Standard with SP2
How can I determine if the ALWT is really 3916?
I executed 'Select * from sysprocesses where SPID>50 and waittime>0'
Which showed;
BROKER_RECEIVE_WAITFOR
waittime=110640ms
waittype=0x0075
status=suspended
program_name=DatabaseMail90 - Id<3780>
Any assistance is greatly appreciated.
Thank You
Magnum
Sql Server Wait Interface?
Jul 20, 2005Oracle has a 'wait interface' for monitoring bottlenecks, etc... does SQLServer have utilities like this?
View 3 Replies View RelatedCheck Condition And Wait
Feb 1, 2007Hi,
I have a data-flow-task that imports data to sqlserver.
Now I want to check, if a special column of an imported record is null.
If yes, I have to wait 10 minutes and jump to the data-flow-task again. (Cjeck and wait).
How can I do this with the integration services?
Thanks
Gerd
SSIS Job To Wait And Run After AS400 Job
Mar 7, 2008
We have a few SSIS jobs that we are currently manually kicking off after we are sure that certain AS400 jobs have run. We want to completely automate this process, so that we don't have to babysit. What is the most efficient way to do this? In the past (on SQL Server 7 no less) I've seen the 400 job setting a flag to 'Y' in a 400 file, FTPing it down to a flat file, and then the SQL job running every five minutes checking the flag. When it was 'Y', the SQL job would run. We do not have the option of using FTP here. Any suggestions would be appreciated! After the job runs, we'd like it to kick off a report as well
SQL Start /wait Command?
May 16, 2008So, I'm fairly new to SQL, and I'm working with a SQL2k5 Database with pre-made packages and what-not. This database was setup before I started this job, and now I'm trying to improve part of the processing in SQL, and so far so good, but I can't figure a couple things out.
The main problem is when I start a SQL command to launch a DTS package from a .sql file, how can I make it wait for the package to complete or fail before moving onto the next part of the .sql script? Hope it's a simple question, I've just taught myself enough SQL to get by in a couple of weeks.
Thanks!
Need To Exec TSQL And Then Wait 60 Sec And Run Again
Dec 14, 2007I am tring to fiqure out how i can run a TSQL and the have it start again 60 sec after it compleats, without me have to push the button. I just need it to loop over and over until my data is deleted. I have to do it this way so my site will still allow customers to login and I need the break so they can. Any help would be great.
View 3 Replies View RelatedAsync_network_io Wait During Replication
Jun 8, 2006I have re-initialize some subscription. After generated snapshot successfully, the synchronization view showing 'The process is running and is waiting for a response from the server.' and I found the replication process is under 'async_network_io wait'. The job has run for more than 8 hrs. Any idea to improve it? Thanks in advaise.
View 3 Replies View RelatedHow To Wait For File Before Continuing
Apr 11, 2007Good morning,
I will start off with the default warning message: I am a beginner. That said, I have an SSIS process that calls an external executable to transform a data file through a homegrown C program. (This will eventually be converted, but for the moment needs to remain.) The end of the run creates a *.done file. How do I use the SSIS tasks to pause/wait, checking periodically, for the existence of this file before continuing with processing tasks? I apologize if this is easy, but I am stumped.
Thanks in advance,
Roger
Info:
SQL 2005
Microsoft SQL Server Integration Services Designer
Version 9.00.3042.00
Compare Incoming Input File Row Values With Database Row Values In SSIS
Jan 23, 2008Hi All,
I receive the input file with some 100 columns and some 20k+ rows and I want to check the incoming input row is existed in the database or not based on 2 key columns. If the row is existed then I need to check all the columns (nearly 100 columns) values in input and the database are equal or not. If both are equal I need to treat them seperately if not there is a seperate logic. How Can I do that check for each row and for each column?
Basically the algorithm is like this, if the input file row is not existed in the database then treat that as new row else if the input row is existed in the database then check all the columns are equal or not. If all the columns are equal then treat that as existing row and do nothing else if some columns are not equal then treat this row seperately.
I found some thing to achieve the above thing.
1. Take the input row and check in the database.
2. If the row is not found in the database then treat it as new row.
3. If row is found in the database then
a) Take the source row and prepare a concatenated string for all the columns
b) Take the database row and prepare a concatenated string for all the columns
c) Find out the hash code for the 2 strings and then compare hash codes for equal.
The disadvantage of this is running a loop 2*m*n times where m is the number of rows and n is the number of columns. It should be done 2 times for input file row and database row.
Can anybody suggest a good method to do this?
What does the function "GetHashCode" for InputBuffer in method "Public Overrides Sub Input0_ProcessInputRow(ByVal Row As Input0Buffer)" will do?
Will it generates hash code based on all the columns values?
Pls clarify.
Regards
Venkat.
The Incoming Tabular Data Stream (TDS) Remote Procedure Call (RPC) Protocol Stream Is Incorrect
May 22, 2006I've read the other posts related to this issue, but I'm just REALLY confused as to whats happening in my case. Like everyone else it was working fine in SQL 2000 but now in SQL 2005 there is an issue. I'm calling a stored procedure with parameters defined like this:
@action varchar(10),
@GLTransactionID int = NULL OUTPUT ,
@GLBatchID int = NULL ,
@GLAccountID int = NULL ,
@CurrencyID int = NULL ,
@LocalDebit decimal(28, 13) = NULL ,
@LocalCredit decimal(28, 13) = NULL ,
@BaseDebit decimal(28, 13) = NULL ,
@BaseCredit decimal(28, 13) = NULL ,
@TransID int =NULL,
@Description varchar(255) = NULL
I am calling this proc from VS.NET 2003 using the .Net SqlClient Data Povider (C#). I'm setting the values of the parameters like this:
cm.Parameters.Add("@action", "insert");
cm.Parameters.Add("@GLBatchID", _gLBatchID.DBValue);
cm.Parameters.Add("@GLAccountID", _gLAccountID.DBValue);
cm.Parameters.Add("@CurrencyID", _currencyID.DBValue);
cm.Parameters.Add("@LocalDebit", _localDebit.DBValue);
cm.Parameters.Add("@LocalCredit", _localCredit.DBValue);
cm.Parameters.Add("@BaseDebit", _baseDebit.DBValue);
cm.Parameters.Add("@BaseCredit", _baseCredit.DBValue);
cm.Parameters.Add("@TransID", _transID.DBValue);
cm.Parameters.Add("@Description", _description.DBValue);
When I execute the call to the stored proc I get this:
"The incoming tabular data stream (TDS) remote procedure call (RPC) protocol stream is incorrect. Parameter 8 ("@BaseDebit"): The supplied value is not a valid instance of data type numeric. Check the source data for invalid values. An example of an invalid value is data of numeric type with scale greater than precision."
Using the VS.NET command window I then inspect that parameter to see what the heck is going on and get this:
?cm.Parameters["@BaseDebit"].SqlDbType
Decimal
?cm.Parameters["@BaseDebit"].Precision
0
?cm.Parameters["@BaseDebit"].Scale
22
?cm.Parameters["@BaseDebit"].DbType
Decimal
?cm.Parameters["@BaseDebit"].Value
1000000
[System.Decimal]: 1000000
So I set a decmial parameter to 1,000,000, that parameter in the DB is defined as decimal(28,13) so should fit no problem, but it seems the Sql data provider is confused and thinks 1,000,000 is decimal (0,22)???
Current Acitvity - Wait Types
Feb 9, 2001When I run some of the quiries, in current activity, wait type would say:
PAGEIOLATCH_SH what does it means ? Is there a source I can see all
types of wait types ?
Help is much appreciated.
Thanks,
Average Lock Wait Time
Oct 3, 2014I keep getting an alert from a Third-Party Tool stating my server is reporting failed components with the following message
Components Reporting Problems:
Average Lock Wait Time(ms)(Critical)
When I look at sp_who2 or sp_lock I don't see anything blocking processes or anything else out of the ordinary.
Chronic CXPACKET Wait Issue
Jul 24, 2007CXPACKET wait type current makes up 63% of all the wait types which is causing latency. I need to identify the specific workloads responsible for waits so I can optimise or MAXDOP them. I already know how to retreived the top IO, CPU, Memory consuming queries but how do I identify the statements and order them by wait time?
Can someone point me in the direction of a command, DMV or will the top CPU list be adequate?
Thanks