Queries :: Deleting Duplicates Without Primary Key?

Nov 4, 2013

I bounced into a big problem with Access 2010 where I cannot seem to be able to remove duplicates from a table containing millions of entries midway through a series of queries.

The table is formed like this:

Date - Serial Number - Reading 1 - Reading 2 - Reading 3
30.7.2013 - 1122334455 - 12345 - 987654 - 654321
30.7.2013 - 1122334466 - 12345 - 987654 - 654321
29.7.2013 - 1122334455 - 12210 - 987654 - 643210
29.7.2013 - 1122334455 - 12210 - 987654 - 643210
29.7.2013 - 1122334466 - 12344 - 986013 - 453213

As you can see, there are some rows which contain exact duplicates and some are a bit different. I wish to remove those fields with exact duplicates and only the duplicated fields. Running "Find duplicates" in Access gives me about 250,000 rows with such data.

I've tried a few options already:
- I cannot use the date as primary key as there are several serial numbers. I cannot use serial numbers as primary key, because there are several dates. Using reading value as primary isn't an option either.
- Microsoft says I should mark all duplicate values with an x and then make a delete query to get rid of all the x-marked rows. For 250,000 duplicates, that's a bit too much manual 'x-ing'.
- If I do a delete query using the Find Duplicates query as a base, it removes all 250,000 entries from that table, instead of just the 125,000 which had a duplicate elsewhere in the table.
- If I make a query which identifies duplicate data and gives me just one row for each duplicate, the delete query still deletes both entries from the original table.
- I could make a new query which would have only unique values using Totals as criteria (for instance, using First for the Date-column). However, this still leaves the duplicate values in the original table. Note that this database is already 800 MB large and new data is imported once a week, for the next decade or so. I cannot have a table get duplicates every week and leave it there.
- If I make a macro which would create the unique values table first and then deletes the old table, what happens next week when I try to import new readings? I would need to make a new macro each time I try to import new data as the table names change. Or is there a way to first run the unique numbers out, then replace the original table with the new one? With a 800MB database, this would put me dangerously close to the 2GB size limit. I wouldn't be able to use this as a part of a macro, as the database would have to do Compact & Repair each time it deletes the original table, midway through a longer series of queries.
- Having duplicates removed from the original import isn't an option either, as it comes in as a overriding excel sheet for the past 3 months, once a week.

As you can see, it's quite a pickle getting the duplicates out of the original table. This is just a small part of a very long macro, which takes about 15 minutes to complete, but due to duplicates the database is getting way too large.

View Replies


ADVERTISEMENT

Queries :: Deleting Duplicates In Non-unique Records

Feb 27, 2014

I have imported a large number of emails into a table tbl_requests.

I had intended to have unique file tbl_requests.date_opened unique, but have ended up with a lot of duplicate records (i.e. tbl_requests.date_opened is not unique !). How to delete any duplicates? I have 15,000 records...

View 3 Replies View Related

Deleting Duplicates

Aug 1, 2006

I have created a table for product codes and prices. After importing data from Excel I have a LOT of duplicated records. So I set up a query to search for 'distinct' records and then set up a delete query to delete anything that is not distinct. The query seems to be setup correctly (I have followed the instructions from Access Help) - but then when I try and run the query I get the message 'Could not delete from specified tables' - Is this a permissions issue? And if it is, where do I set up the permissions for the database? Thank you any ideas would be greatly appreciated!

View 1 Replies View Related

Deleting Duplicates

Feb 11, 2008

my problem is this:
i have a table that has three data feilds
id, amount1, and, ammount2
i want to allow the user to imput duplicates, but i want my database to delete all but one of the duplicate feilds, and add the ammounts of the deleted records to the ammounts of the non deleted duplicate...

View 3 Replies View Related

Deleting Duplicates

Nov 14, 2004

I 've created a query using the access wizard that finds duplicate records. It works fine. But then I would like to delete the duplicate records and keep just one(the first) . The problem is that the duplicate query returns the amount of duplicates found, not WHICH are they so I can delete them. Is there any way to alter the duplicate query , or to create a delete query using the duplicate query to achieve this? Or any other idea? Thank you in advance

View 1 Replies View Related

Deleting Duplicates Query.. With A Twist

Jun 17, 2005

Hi,

Been reading the forums for a while now. Thanks to all you knowledgable people out there! Been a huge help in learning this program.

Here's a question that stumped me...

I have a table that is about 90,600 entries long with 4 fields per record.

Now here's the issue:

Approximately 25,000 of these entries are duplicates - based upon TWO of the fields.

SO we have:

Field1 Field2 Field3 EntryDateField

if Field1 and Field2 Match, I want to delete the record that has the OLDER entry date.

any ideas how I might accomplish this?

Thanks!!!

J

View 4 Replies View Related

Is There Any Way To Make A Primary Key Contain Duplicates??

Mar 3, 2006

hi,

is there any way to make a primary Key contain duplicates??

any ideas ??

Many thanks

View 3 Replies View Related

Cannot Remove Duplicates Using Primary Key

Mar 27, 2006

Hi all, total newb here with a question i can't find answered anywhere. (you are my only hope) This database contains over 70k records and is a collection of user registrations over the years. Heres the issue.

After running the find duplicates query and get my list (over 8000 dupes) i get a sample like below. I cannot now run the append query as every other site or article says to. I can't set the primary key to surname because there are so many records that have the same last name but diff. first name. I need it to remove the dupes based on the EditDate, keeping the newest record.

List before dedupe:eek:
IDTag..................Surname......First......... ......EditDate
37659362.............Adams.........Mary........... ...2005.10.21
57506..................Adams.........Mary......... .....2005.04.25
37661858.............Adams.........Nichole........ ...2005.06.23
37661857.............Adams.........Nichole........ ...2005.06.23
37661795.............Adams.........Nichole........ ...2005.06.17
51176..................Adams.........Nichole...... .....2005.03.16
37657399.............Adams.........Rudo........... ...2006.02.15
54640..................Adams.........Rudo......... .....2005.02.08

List after dedupe:D
IDTag..................Surname......First......... ......EditDate
37659362.............Adams.........Mary........... ...2005.10.21
37661858.............Adams.........Nichole........ ...2005.06.23
37657399.............Adams.........Rudo........... ...2006.02.15

Any suggestions?

Newb

View 4 Replies View Related

Deleting Multiple Tables Or Queries

Mar 6, 2007

Hi.

I have a cluttered database with too many 'no longer necessary' queries and tables.

Is there a way (other than deleting them one at a time) to delete multiple ones at once, as I can in My Documents, et cetera?

Russ

View 2 Replies View Related

Queries :: Deleting Fields Within A Table

Sep 16, 2014

Is it possible to create a query that will delete the fields within a table?

View 1 Replies View Related

Deleting Forms,queries,macros In Another Database

Feb 16, 2005

I have a list of objects (forms,queries,macros,reports the number varies but could be up to 100) that need to be deleted from several other databases (about 20).

I could do this by hand (go into each database and select and delete each object) but this is slow and error prone.

I could create a macro and import it into each database and then execute it (I know how to do this)

or (!!!!!!)

there should be some way to be in one database (HOME) and programatically (VBA) delete in the other database (OTHER).

I know how to

Dim db As Database
Set db = DBEngine.Workspaces(0).OpenDatabase("c:other.mdb")

What do I do next. I could use the db.execute command for tables (which I don't want to delete) but what about forms,queries,macros, and reports ???? I tried to modify the MsysObjects table but ran into permission problems.

Any Ideas

View 1 Replies View Related

Queries :: Deleting Data From Selected Records

Sep 23, 2014

My Membership Database includes 3 fields in the main Table which I need to clear prior to the new subscription year, but on a selective basis.The fields are R/N for (Renewal or New), a subscription date and an amount field (Currency).

At the due date I need to clear all 3 fields except where N/R=N AND date >01/01/2014, when data in all 3 fields is to be retained.I have tried several SQL codings to achieve this but have been unsuccessful so far.

View 6 Replies View Related

Queries :: Deleting Records Based On Selections From Another Table

Apr 1, 2013

I am trying to create a Delete query.

I am trying to delete a specific part from multiple BOMs in my database.

I have a table of the BOMs that I want to look in. I called this table PartTable. I also linked my database table SYSADM.REQUIREMENT which contains all the requirement parts for all of our BOMs.

So I am wanting to delete only part number 123XX from each of the BOMs in my PartTable.

I am able to select the records with:

Code:
SELECT SYSADM_REQUIREMENT.*
FROM SYSADM_REQUIREMENT INNER JOIN PartTable ON SYSADM_REQUIREMENT.WORKORDER_BASE_ID = PartTable.PART_ID
WHERE (((SYSADM_REQUIREMENT.WORKORDER_TYPE)="M") AND ((SYSADM_REQUIREMENT.WORKORDER_BASE_ID)=[PartTable].[PART_ID]) AND ((SYSADM_REQUIREMENT.WORKORDER_LOT_ID)="0") AND ((SYSADM_REQUIREMENT.PART_ID)="123XX"));

Now how do I delete these same records.

I am getting error saying I have to select a table to delete from....

View 11 Replies View Related

Queries :: Delete Duplicates In Row Of Data

Feb 10, 2014

I need to delete duplicate rows (or create new table without them) but the duplicate is not the entire row of data.

Date Time LeagueMatchingId League HomeMatchingId HomeName AwayMatchingId AwayName
HomeScore AwayScore HomeRedCardCount AwayRedCardCount FTOU FTOverOdds FTUnderOdds
20121202 10/02/2014 20:34:02 17 FRANCE LIGUE 1 147 Lorient 154 Toulouse 0 0 0 0 1.5 0.88 -0.98
20121202 10/02/2014 20:34:02 17 FRANCE LIGUE 1 147 Lorient 154 Toulouse 0 0 0 0 1.5 0.88 -0.98

[Code] .....

In this example I am only interested in the first and last rows, since the ones in between have the same data in the last few columns. They are not true duplicates since the time stamp for each is different.

Currently thinking I need to compare each and every row to the prior row in VBA and delete if criteria match.

View 4 Replies View Related

Queries :: Finding Duplicates Using Last Name And First 3 Letters?

Apr 2, 2014

I need to create a query that will pull duplicate names out of my db.

I would like it to pull all names that have:

duplicate LastName and duplicates of the first 3 letters of the FirstName.

For example, if I had the names:

Bland, Abe
Brown, Abe
Brown, Bill
Buster, Jon
Buster, Jonathon

I would like my query to return only Buster, Jon and Buster, Jonathon.

View 4 Replies View Related

Queries :: Matching Values Without Duplicates

Oct 23, 2013

I'm looking for a way to grab values from one table that match a criteria, and append them/update them to another, but only grab one value once. Example:

Part ID ---- Profile Profile ID ---- Locations
Part1 - Profile 1 Profile 1 ---- Location 1
Part2 - Profile 1 Profile 1 ---- Location 2
Part3 - Profile 1 Profile 1 ---- Location 3

I have a database of parts that fit in certain profile sizes. Each profile size has multiple open locations. I need a query to grab a location, assign it to a part, grab a new location, assign it to the next part, etc, without assigning duplicate locations to different parts.

Right now, I'm doing this by using the First/Last aggregate functions, marking them as used, and looping it again and again until all have unique locations. Is there a better way to do this?

View 8 Replies View Related

Queries :: Count Duplicates By Month?

Jun 4, 2015

I want to be able to find duplicate numbers in the "claim #" field that exceed 20 within the "date of service" field for a month. I have a date box which allows me to select data from only one month at a time or all. What I'd like to be able to do is group the results by month. Currently it is only matching numbers by day not month.

SELECT HMR.[Claim #], Count(HMR.[Org #]) AS Total, HMR.[Date of Service], Count(HMR.[Date of Service]) AS [CountOfDate of Service]
FROM HMR
GROUP BY HMR.[Claim #], HMR.[Date of Service]

[Code] ......

View 14 Replies View Related

Queries :: Identify Duplicates With Three Criteria

Apr 5, 2013

I have the following columns in my table 'Catalogue'.

Contract_contractitem
contract
Supplier name
Item number
Short description
Long description

I need to identify duplicates if the short desciption is the same, long desciption is the same but the item number is NOT the same.

e.g.

4700000551_978, 4700000551, FISHER SCIENTIFIC BIOTECH LINE ApS, mbp-6014, 0,2 ml PCR Tube, Hotstart, wax bead, NS, 0,2 ml PCR Tube, Hotstart, wax bead, NS 96 tubes/pack, 10 packs/Cs

4700000551_977, 4700000551, FISHER SCIENTIFIC BIOTECH LINE ApS, mbp-6010, 0,2 ml PCR Tube, Hotstart, wax bead, NS, 0,2 ml PCR Tube, Hotstart, wax bead, NS 96 tubes/pack, 10 packs/Cs

View 4 Replies View Related

Queries :: Hide Duplicates From Query?

Aug 1, 2013

I have a query showing duplicates, I want to hide the duplicates that have a null value in one of the rows..

Trunc_Name M_UNDERLYIN MaturityDate Duration SumOfSumOfM_VEGA Volatility
SPX P1400 DEC13 INX 21-dec-13 4,766666667 -108 812,00 0,22437
SP6652 CALL SP6652 04-okt-17 50,86666667 124 376,00 0,21
SP6652 CALL SP6652 04-okt-17 50,86666667 124 376,00

Here's an example of the a duplicate, all 4 first rows are duplicates, but the last value is null for the last duplicate. I want to hide the one the duplicate that has null value!

View 4 Replies View Related

Queries :: Total Excluding Duplicates?

Aug 30, 2013

I have a table with the following fields:

Customer
Month
Qty
Unit Price
Extended Price

I want to know, by month, what the total extended price is for all customers. But some customers appear more than once in a month. So I also want to know the total extended price for the month, but I only want to include the extended price for each customer once, using only the record with the highest unit price (or extended price, qty for each customer is consistent), and ignoring additional records with the same customer in that month.

I am an excel user, and I could quickly do this with pivot tables... but I have too many records for Excel to handle this efficiently (it just keeps crashing!), so I thought I'd move to access... but I don't know how to use access! So I may need a little hand-holding here.

View 10 Replies View Related

Queries :: Join Two Tables With Duplicates

Oct 19, 2013

I am trying to combine two peculiar tables in Microsoft Access and have been unable to do so even after doing a lot of brainstorming and searching on the internet.

The two tables are spend and export
Spend
+-------------+--------+-------+-------+
| Country | Metal | Month | Spend |
+-------------+--------+-------+-------+
| China | Iron | Jan | 100 |
| China | Iron | Feb | 200 |
| China | Iron | March | 300 |

[code]...

View 2 Replies View Related

Find Duplicates And Create Table With Dates Of Duplicates

Feb 12, 2008

Hi this is my first post... so hi all :)

ok what i have is a table with contact details 900k plus

there are about 90k of which are duplicates.

this is the basic feilds that are important in this case.

Id, data_source, data_recived, data_code,

what i want is to have a table with unique records (no dups in data_code)

this table will look like this...

Id, data_code, Num_dups, dup1_source, dup1_date, daysbtw_Dup1_dup2, dup2_source, dup2_date, daysbtw_Dup2_dup3 ,dup3_source, dup3_date, daysbtw_Dup3_dup4 ,dup4_source, dup4_date,

I know there is no more than 4 dups of each record.

what i want from this is a table that will give me a record of how many dups for each record then all the dates that they were added and the date between each record entry.

if anyone can help it would be great .

thanks in advance.

View 6 Replies View Related

Form Based On Queries - Deleting Field From Main Table

Mar 28, 2014

In my Access2000 db, I have a table, on which many different queries are based. Many forms are based on those queries. I want to delete some redundant fields from the table. I tried deleting one, but I found I could no longer open any of the associated queries and forms. Is there a simple way to delete table fields so that it doesn't stop me from opening associated queries and forms? (There are hundreds of them, and I have a lot of table fields that I want to delete).

View 1 Replies View Related

Queries :: Remove Duplicates Entries In Table?

Jun 11, 2013

I have a table called Stock Levels which contains 3 fields. (ID, ProductID, StockLevel) ID is the Pkey, ProductID contains duplicates and StockLevel which contains different stock levels

and I am trying to remove the duplicates and retain the the data so I am left with the correct stock number

what I have done is the following, but I am still getting duplicate values in productid and stocklevels

SELECT DISTINCTROW id, productid, stocklevel into mynewtable from stocklevels

I have attached a screenshot of the table

View 3 Replies View Related

Queries :: Using Append Query But Not Adding Duplicates?

Jul 30, 2013

I would like to use an append query that appends data from a union query into a table, but doesn't add duplicates based on a certain field.

I have been trying to use this code, but it doesnt seem to work.

INSERT INTO [Cotton13/14]
SELECT DISTINCT [Contracted Farmers].*
FROM [Contracted Farmers]
WHERE NOT (SELECT * FROM [Cotton13/14] FROM [Cotton13/14] AS [Cotton13/14] WHERE [Cotton13/14].[OF Codec] = [Contracted Farmers].[OF Codec])

Cotton13/14 is the table I want to append to and Contracted Farmers is the union query. However the code is giving me an error.

View 7 Replies View Related

Queries :: Remove Duplicates - If 2 Fields Are Equal

Nov 19, 2013

I have a large table with many fields and many rows. There is no primary key. I'll call one field ParentPN, and another field ChildPN. There are many other fields as well. I want to identify all rows where BOTH the ParentPN and ChildPN occur more than once. I know how to create a query to identify duplicates of ONE field in the table, but not two. I can solve this with VBA: I will read the two fields of interest in the first row, then compare both values with every other row. If it finds another row with BOTH ParentPN and ChildPN identical with the first, that's a "hit". Then, repeat with all the other rows. I could find ways to make this run faster, but I was wondering if there are any build in functions to accomplish this. I looked at the Find Duplicates query builder, and all I see is I can select ONE field to search for dupes, not two.

View 14 Replies View Related







Copyrights 2005-15 www.BigResource.com, All rights reserved