Large Unused Tables Vs Performance
I've been working on a website with a rather large a database. One of the about 100 tables in there was about 1/4 of the total database size.
The table had no use anymore at the time I worked the website, so it was just sitting there, doing absolutely nothing. Obviously to spare diskspace we deleted the table.
But the question I have now, can unused large tables affect the perfomance of Mysql?
I've searched a bit around, but honestly I couldn't find a clear answer about it right away (maybe I haven't searched enough either :) )
View Complete Forum Thread with Replies
Related Forum Messages:
Large Table Performance Problem With MYSQL Performance
I am doing some benchmarking of an application using MYSQL under Windows and I confused about the results. Environment Emachine 6417 with amd3200+ 64 bit processor 3.3GB ram mysql 4.1 with odbc 3.51 MYISAM windows 2003 server std edition date, account and invoice number are indexed Database size 18 million rows I am querying (selecting) columns of a date and an account our tester program that opens a socket to the Mysql database and does a select for the above n times each time the date and the account is randomized to minimize hits on records closeby. This program will perform over 1000 queries per second. At the end, the Mysql socket will be closed Even when the socket is closed each time, I still get 400 queries per second. When I enter a similar query manually a web interface, I get about 3-9 second response time. This program opens/closes a socket for each query when using EMS I get similar 9 second results. Does anyone have any suggestions Also in production, this table will be accesse for both read and write will I have problems. My testing showed that Innodb is much slower.
View Replies !
Update Performance On Large Table
I have a table that is approaching 4 gigs. I have optimized as much as I can with indexes so that select statements are ok, but updating entries seems to be taking a bit of time. I have 2 questions: 1) I know about the EXPLAIN command for select queries, but is there something like that for updates? I get an error when I try to do "EXPLAIN UPDATE mytable..." Do updates use indexes the same way as selects? 2) Would breaking the table up into seperate tables and then using a merge table increase the performace? I've heard that key reads are slower on merge tables - but since I could make 4 tables of a gig each, wouldn't it be faster opening up 1 smaller file to make an update instead of a huge one?
View Replies !
Very Large Data-files - Performance?
I have to handle a large amount of Data (about 5GB) in one table. I want to extract the essence out of this table which should be something like 1MB and save it to a new table. Mostly used commands are GROUP, WHERE, BETWEEN, IN and LEFT JOIN. I use 3-4 temporary Tables. The script is ready and tested on a file of 60MB (750.000 datasets) and works fine (needs about 90seconds). What I want to know is: Can I expect the query to work through my big data within some ours, or will I fail with a standard PC? What can I do to optimize (Hardware/Code)? There is only one query to work through at one time.
View Replies !
Mysql Performance On Large Joins (its Bad)
Mysql is unable to perform joins with large tables ON VARCHAR. these joins run fine when the queriers are performed on INT values. However int values are not practical in this instant because I am joining on product SKUs. sample SQL ----------------- SELECT a.sku AS this, b.sku AS that, b.price, b.SRP FROM table1 a INNER JOIN table2 b ON a.sku LIKE b.sku TABLE specs ------------------------- table1 14000 records table2 13000 records Running Mysql 4.1, PHP 4.3.
View Replies !
MySQL Tables Performance
I have a table where misc data are stored. Right now this table has about 30 columns, but for sure it will be more in the near future. So I wonder how the big number (50-100) of table's columns affect for MySQL DB performance. Maybe it's better to create more tables rather than more table's columns ??
View Replies !
Performance Affected By Tables
I'm develloping a game which will have to store information about objects located in each square of a large grid. Since there can be many, or none, or a combination of different objects in any given square at a time, I figure I should create a table for each square on the grid containing the names of the objects in it. This may lead up to 1000-5000+ tables though, many of which could/will be empty, and I was wondering if that is the best approach I can take?
View Replies !
Tables Structure - Performance
I need to design a table structure for a new application. We are using MyIsam tables. I am not sure which approach is better: a simple structure with 2 or three hudge tables or a more complicated structure with 6, smaller, tables. I see the benefit of the first way that because I will be able to get more info with less queries. I also see the benefit of the second way because it will better represent the data and also maybe because the tables are smaller it will be faster then quering a hudge table.
View Replies !
Performance Of A Database With A Lot Of Tables
I am working on hosting solution for a popular open source script, where users can sign up and get an own forum. We use InnoDB, and each user has an own set of tables (around 30); there are currently around 500 users registered. Despite the large count of tables, database is small, around 50 MB. Server is dual Xeon with 4GB of RAM and MySQL uses all memory, available on the server, so there's even nothing left for Apache. As the user base is growing, what steps would you suggest me to do, in order to keep good database performance and make it more efficient on RAM usage? The former is more important.
View Replies !
Index Unused
SELECT COUNT(*) AS `total`, ROUND(COUNT(IF(DATEDIFF(CURDATE(), lastModified) <=7,1,NULL))*100/COUNT(*),1) AS `pnt`, FROM example GROUP BY col1 ORDER BY `total` i have this table with lastModified column indexed, but the above call still use full table scan, how can i reconstruct this query to use index,
View Replies !
Large Tables
So i've got a table with 15 million rows. id is the primary key SELECT COUNT(id) FROM listings WHERE id != 0; Takes about a minute to load... which is insanely slow.. anyone know what could be causing this?
View Replies !
Very Large Tables
I am looking at using mysql to store some particle physics data. So the idea is (for example) I would have a table with a list of events, then a table with a list of particle types (one event can have many particles) and so on. The problem is that quite often I want to calculate something for every single event, but this table might run into 10s of millions of events and many GB. Obviously it would be mad to load the whole result set into memory then iterate over it. My question is: is there any way to have the db push results to the users one (or a few) at a time in an effiecent way? I was thinking of something like an iterator that pulls the data from the disk one (or few) at a time. An alternative (but much less useful) method might be to hand the function I want to calculate to mysql, for example: select myfunc(x,y,z,a,b,c) from events where energy>3; But as far as I know there is no way of defining your own functions in SQL
View Replies !
Large Table Or Many Tables?
I posted a similar question about 6~12 months ago with no definate answer, so I will give it another go, as the choice has popped up again in my programming life: If the number of entries is the same is it more efficient (better) to have a single table with many columns, or many tables with few columns? The obvious answer is a single table, because there is less closing and opening tables, but the reason why I ask is what about searching through a lot of columns being even more inefficient? The situation is I need to record data for every week of the year. Say ~50 columns per week --> 2600 columns. Or, do I split it so it's 52 tables with 50 columns each? In each of these options the user (or user ID) will be the primary key. The seemingly pointless other option is to insert new columns with every new user, and have the week-item as the primary key?
View Replies !
Large Number Of Tables
I would like to ask whether it is feasible (or whether it makes sense) to run a PHP script that creates a number of tables for each user. Basically, this script allows users to sign up for a free service, and for each user that signs up, the script creates 9 tables in the database to contain that user's details. Is this a sensible way of containing user data, and what are the inherent problems this may cause especially when say are 1000 users signup?
View Replies !
Large DB Should Use Multiple Tables?
I have a social networking site similar to myspace, currently my table which links who is who's friend on my site is 2 weeks old and is at 300,000 rows but is only at 15mb in size, this is the most accessed table of all my tables. My question for performance, as this table grows should it eventually make a new table? If so here is my current select code select userid,friendid,status from friend_friend where userid='$temp[auto_id]' and status='1' Could someone give me an example of how o select this from multiple tables if I add more for this table in the future?
View Replies !
Managing Large Tables
I recently inherited a database (version 5.0.18) that has 1 table constantly growing out of hand between 10GB -30GB, therefore making is difficult/impossible to query. Also, the Archive Storage Engine is not installed. Currently, the table is manually renamed (i.e. tablename_daterange) and a new table created. ....
View Replies !
Slow Performance :: Navicat To Import Tables From An ODBC Connection
have been using Navicat to import tables from an ODBC connection I have. The import goes extremely fast but once the data is locally stored and I try to manipulate it by joining tables, the program goes slower and sometimes even crashes it. If the same data is however imported into a table in MS Access, and a query is created it runs fine. I would like to know if there is a way to improve the performance of my MySQL database as the reason I was using it is because of its scalability. I was wondering if there was something I was doing wrong or could do better. The Server the data is downloaded to right now is has the following specs: Windows XP Pro Intel Core 2 CPU 2.13 Ghz 2 GB RAM
View Replies !
Join Tables On A Large Database 200 Meg
i am trying to work out the most efficient way to list say multipl= e categories of entries, the database is quite large about 200 meg.=20 I would like to know if using join tables is more efficient than storing th= e keys in a varchar field then within the second loop doing a where in (1,2= ,3,4) where the 1,2,3,4 are the stored category keys in the varchar field = rather than a where in (1), where 1 is the pirmaryID of the entry for insta= nce ?
View Replies !
Optimizing Mysql For Large Tables
I have been working with Mysql for about 5 years - mainly in LAMP shops. The tables have been between 20-100 thousand records size. Now I have a project where the tables are in the millions of records. This is very new to me and I am noticing that my queries are really slowwwwww! What are the options that I have to speed my queries on the mysql side with regards to the my.cnf file. I have a fair understanding of sql optimization and I understand explain. I just want to see if there is something that I can do with the server also.
View Replies !
Slow Join On Large Tables
I have two tables: D (500,000 recs), and DL (2,500,000 recs) D has a PK and an index on HLQ. DL has a PK and an index on ID. The following SQL: SELECT HLQ as "HLQ", count(*) FROM D, DL WHERE D.DLID=DL.ID GROUP BY HLQ produces the following explain: tabletypepossible_keyskeykey_lenrefrowsExtra DALL500000Using where; Using temporary; Using filesort DLeq_refIDID4D.DLID1Using index The query takes ~ 3:30 on a Athlon xp2200; 1GB RAM; default bufer settings. Adding the following buffer settings only slightly decrerased the time (~3:00). key_buffer=512M table_cache=256 sort_buffer=16M read_buffer_size=16M It appeasrs that the 'Using filesort' on table D is due to the Group By clause and is the problem. I have an index on HLQ. Is there any way to get MySQL to use it?
View Replies !
One Large Table Or Many Small Tables?
I'm trying to decide whether to use one large table or many small tables. I need to gather information from various devices (about 500). Each device has its own Id and some data. Should I use only one table with an indexed column for the ID and another column for the data, or should I use 500 tables each with only one column for the data?
View Replies !
Large Tables, Very Slow Deletes.
I've been using MySQL for a while now and are starting to run into limitations. Either my own, or something else. I have this really large table, it stores images, it has a mediumblob field, an unsigned id integer field and a timestamp field. This table contains up to a few million rows and is constantly filled real time. It grows up to sizes between 50 and 100 gb and in the future probably even larger. This data is not supposed to be stored for all time, therefor when a certain treshhold is reached, the oldest 5000 rows are deleted to make room for newer rows. This is where my problem kicks in. Whenever I try to delete those rows, it can take for ever to complete. 500 rows I can delete without problem, 2500 becomes slow, 5000 rows takes several seconds and 10.000 rows or more makes it looks like things are frozen. Now the temporary solution I have is to delete 500 rows at time, but this means I must do that every 10 seconds just to maintain status quo. I would much rather check every 5 minutes or so, and if needed delete 15.000 rows in one go. I use MyISAM tables and tried setting the key size to 128M, no luck. I run this on a dual Opteron system with a GB of memory and WinXP-SP2 Proffesional.
View Replies !
One Large Table Vs. Many Small Tables
I'm working on a design using PHP & MySQL and I'd like to get some opinions on this. My design has several tables that will be referenced but I'm wondering if those tables should be broken down even more and referenced more dynamically. The reason that I wonder about this is for long term goals. I hope that eventually there will be two or three thousand records that will be used on a regular basis. These records will need to be separated into groups, but I'm not sure if I should use a field in the database table or create a new table for each group. If a few hundred records could be in each group, do you think it's better to use one large table with a field for the group ID, or a new table for each group?
View Replies !
Large Number Of Tables And Columns
i'm developing an online portal, which my MYSQL database contains 100 tables.. my doubt is if any table contains 30-40 columns will it become a problem? like my html front end form has 40 fields for data insertion. and i'm not maintaing any duplicate data if my DB contains 100-150 tables will it create any problems? as i'm inserting data from EXCEL via PHP to MYSQL, please suggest me whether a healthy MYSQL table is dependent or independent on NO:OF COLUMNS(FIELDS) or not?
View Replies !
Calculating Unused Time, Per Day Given Date Ranges
I've found a few potential solutions to this, but mostly they use Oracle Analytics syntax or SQL server specific extensions. I'm struggling to come up with something that works in Mysql. I've got a table of date ranges, let's call it "bookings" Code: roomid | startdate | enddate -------+------------------+----------------- 1 | 2008-02-03 13:00 | 2008-02-03 17:00 1 | 2008-02-03 18:00 | 2008-02-03 19:00 I'm trying to come up with a query that will give me the number of unused minutes for a room for a given day, such as: Code: roomid | date | unused -------+------------+------- 1 | 2008-02-02 | 1440 1 | 2008-02-03 | 1140 1 | 2008-02-04 | 1440......
View Replies !
Speeding Up Large MySQL MyISAM Tables
I'm somewhat new to MySql. I've been using it for a while, but pretty much out of the box setup, and am starting to suffer heavily with my larger tables. I have a table with 5,000,000+ rows that I have to search and do joins on. Although I have an index set up for it, the joined select will still take some 400+ seconds to return, which is obviously unacceptable. This is due to enormous HD access. Perhaps someone can help me with indexes here; I was under the impression that the index for the tables are cached in memory, and therefore permitted "instant" searchability, without having to retrieve data from each of the rows of the DB. Is there a startup parameter, or something in the mysql.ini file, that must be set to allow for this? I have the index configured properly, and have made sure that the query uses there parameters in the where clause in the same order that they appear in the index. Code:
View Replies !
Speed Of InnoDB DELETEs On Large Tables
I am finding delete queries on large InnoDB tables very slow - are there ways to speed this up? I have a table with about 100 million rows: I am trying to delete just a few of these rows (the following select takes a couple of seconds): [color=blue] > SELECT count(*)[/color] -> FROM UserSnap -> WHERE LogDate<now() - INTERVAL 750 DAY; +----------+ | count(*) | +----------+ | 308969 | +----------+ [color=blue] > DELETE FROM UserSnap WHERE LogDate<now() - INTERVAL 750 DAY;[/color] That delete query takes hours to run. The structure of the table is: [color=blue] > desc UserSnap;[/color] +----------+-------------+------+-----+---------------------+-------+ | Field | Type | Null | Key | Default | Extra | +----------+-------------+------+-----+---------------------+-------+ | LogDate | datetime | | PRI | 0000-00-00 00:00:00 | | | Period | tinyint(4) | | PRI | 0 | | | UserName | varchar(50) | | PRI | | | | RateType | varchar(50) | | PRI | default | | | Rate | float | YES | | NULL | | +----------+-------------+------+-----+---------------------+-------+ Any suggestions on why this is slow, and what to do about it?
View Replies !
Loading Large Tables From Files (wikipedia)
I'm an Oracle DBA and new to MySQL. I was trying to load a 5.6 GB xml file onto mysql database using mediawiki tools. The performance decreased gradually and crashed at ~60% completion. Now I'm planning to use xml2sql tools to convert the dumps to txt format. Then I'm planning to load this using mysqlimport. I would like to run this load faster, and on Oracle, I would use the IMP parameters such as INDEXES=n and so on. What are the recommended steps when using mysql? Should I target any of the system variables? I have a 4G ram on the server and can use all of it for the load. Any pointers?
View Replies !
Time Out Message When Query Large Tables
I'm trying to get data from 6 large tables but the volume of data in each table is too large and even select * from one of them make the system stop. I have afew questions: 1-what can I do to avoid the system stop or time out message? 2- To use several tabels infomation should I use 'View' command or can I use other methods? 3-I need to create a new table and insert the result from query in it. If I use the "view" can I insert the result of the view in a table? (I use postgresql).
View Replies !
Selecting From Multiple Large Tables Quickly
My skills with MySQL typically end at "SELECT * FROM table WHERE stuff", so I've had a lot of trouble with optimizing this query. I've tried doing multiple select statements for this, but that usually comes out with an average execution time of 99s. This method has a execution time of 6s still, so I'd like to get it down. I have 4 important tables here (I'm only posting the columns which I use): `auth` UID mediumint(8) unsigned PRI NULL auto_increment username varchar(32) MUL clan_id mediumint(8) unsigned MUL 0 clan_abbrev varchar(7) MUL `clans` ID mediumint(8) unsigned PRI NULL auto_increment Abbrev varchar(7) MUL Name varchar(64) icon_id smallint(5) unsigned 0 `stats` UID int(10) unsigned PRI 0 kills mediumint(9) NULL `usernames` UID int(10) unsigned 0 ID int(10) unsigned PRI NULL auto_increment Name varchar(32) Uses mediumint(9) 0 `auth`, `stats`, and `usernames` are linked by `UID`. `clans` is linked to a row in `auth` based on it's `clan_id`. I need to search the `usernames` table for a `Name` which matches a certain text (I use "storm" in my example below). I need to return a result with matches to that search, with each row containing:The `UID` which links the tables together for that user.The matching `Name` from `usernames`The matching `username` from `auth`The `Uses` from `usernames`.The `clan_id` from `auth`.The `Abbrev` from `clans`.The `Name` from `clans`.The `icon_id` from `clans`.The `kills` from `stats`. The result should be ordered by `Uses` (from `usernames`) -- highest to lowest. The tricky part is that some users may have Ɔ' as their `clan_id` in the `auth` table, in which case the clan id, abbrev, name, and icon_id should all be blank (0 or '' based on the type). Here's what I managed to hammer out with my limited knowledge of SQL: ( SELECT usernames.UID AS `UID` , usernames.Name AS `result` , auth.username AS `playername` , usernames.Uses AS `Uses` , auth.clan_id AS `clan_id` , clans.Abbrev AS `clan_abbrev` , clans.Name AS `clan_name` , clans.icon_id AS `clan_icon` , stats.kills AS `kills` FROM `usernames` , `auth` , `clans` , `stats` WHERE usernames.Name LIKE '%storm%' AND auth.UID = usernames.UID AND clans.ID = auth.clan_id AND stats.UID = auth.UID ) UNION ( SELECT usernames.UID AS `UID` , usernames.Name AS `result` , auth.username AS `playername` , usernames.Uses AS `Uses` , 0 AS `clan_id` , '' AS `clan_abbrev` , '' AS `clan_name` , 0 AS `clan_icon` , stats.kills AS `kills` FROM `usernames` , `auth` , `stats` WHERE usernames.Name LIKE '%storm%' AND auth.UID = usernames.UID AND auth.clan_id =0 AND stats.UID = auth.UID ) ORDER BY `Uses` DESC It works great, but the average query takes around 6s (these tables have several thousand entries in each).
View Replies !
Can I Start AUTO_INCREMENT Back To Fill In Unused Numbers?
I've got an InnoDB table of people who are identified by a PersonID (primary key), and it's passing 50,000 and growing quickly. My users have deleted thousands of records in the past, so there are thousands of unused PersonID number's that can be used that are less than 50,000, where it is now. Can I set the AUTO_INCREMENT back to, say, 1 and have it fill in the empty numbers? Is that even possible? Will it just look for the next available, unique number and use it?
View Replies !
IO Performance
I'm running a 5.0.51a mysql server on pentium M. My application is made of 30 myisam tables written every 2 seconds and a very few reads (1 evry hour) from the whole table. Each table is a kind of circular buffer (fixed number of line, current line). Each line of a table contains a medium blob whith fixed size (about 70k). I know the average output (Kb/sec) from the client to th server, says 200 Kb/sec. When trying to measure this flow with 2 invocations of "show global status like '%Bytes%'; followed by division by the elapsed time, I find between 1 and 2 times more (400 Kb/s). When I measure the output to the disk with the iostat command (iostat -d -k 10) I find 4 times more (800 Kb/s) and a lot of read Kb/s (100 kB/s). Is this normal or is there some tunning to do ? During this time, the iowait rate is very high (30%). Is this normal or is there some tunning to do ?
View Replies !
Best For Performance
It's a web application, accessed via internet, with around 100.000 clients and each client have around 10 users. This web app have a database of around 100 tables. Ok, the question is: Is it better to have each client with it's own catalog on the database (each catalog with around 100 tables) or is it better to use one catalog for all the clients (the tables would have a lot more records this way, but only one catalog)? I'm considering performance.
View Replies !
Which Is Better In Performance
country (id) country (1) America (2) Germany (3) Canada (4) China (5) Egypt (6) Russia (7) Korea (8) France (9) Brazil city (id) city (1) Berlin (2) New York (3) Cairo (4) Paris (5) Seoul (6) Peking (7) Chicago (8) Moscow (9) Denver I have two tables like the above. And I have one of these tables below. message1 (id) (city) message (1) (2) I love New York. (2) (5) Kumkang mountains are beautiful. (3) (4) I was born in France. (4) (6) Great Wall is great. (5) (7) Chicago is a big city. message2 (id) (city) (country) message (1) (2) (1) I love New York. (2) (5) (7) Kumkang mountains are beautiful. (3) (4) (8) I was born in France. (4) (6) (4) Great Wall is great. (5) (7) (1) Chicago is a big city. code1 select city, country from message1 left join city on city.id=message1.city left join country on country.id=city.country code2 select city, country from message2 left join city on city.id=message2.city left join ccountry on country.id=message2.country I think The codes above have a same result.(sorry, I didn't test it.)
View Replies !
Performance Of LIKE
I'm weighing the performance of materialised path vs. nested set algorithms, for storing a tree structure. The tree will have a lot of searches, a fair amount of inserts, but probably fairly few move-operations. The decisive factor seems to be the speed of a LIKE-query. I know, that regular expressions perform pretty well, as long as they are anchored. From this, I deduct, that LIKE must be able to perform equally good, since a path is indeed anchored. The questions is, if a LIKE will always result in a full scan, or if it can utilise indices? If it doesn't allow use of indices, would it be worth it, to use an equality comparison, on a substring (Which is essentially, what I'll be querying for) or am I just entirely missing the point?
View Replies !
Innodb Performance (CPU, RAM)?
I have a table with approx. 150,000,000 ++ rows. It should be approx 80 byte for each row. Now I'm planning to use it with mysql/innodb. Can you guys suggest anything about this. I have many questions... Can Innodb work well with such tables? Will queries (select/update) be slow? What hardware do you suggest, cpu/ram?
View Replies !
Balance Performance
My computer is AMD athlon 64 3400+ with 1.5G DDR. when i execute the following sql in mysql 5.XX. select c.filename, c.topic from articles c where c.topic in (select a.topic from articles a group by a.topic having count(a.topic)>1) order by topic; It eat up all the cpu power , and the mysql server stop responsing until end of this query. Even my apache stop responing too. How to fix this? How can i set the CPU limit of each query connection?
View Replies !
Performance MySQL With ADO VB
I'm developing an VB application which uses the SAX parser and ADO VB to insert.update data in in MySQL. The perormance is very bad. If I use the same data and insert/update to a MS Access database it takes 14 minutes. But using MySQL it takes about 5 hours! Here's the piece of code which takes the most time. Does anyone have an explanation for this? Code:
View Replies !
MySQL Performance
We ran MySQL (4.0.15-standard, 64-bit, SuSe 8.0) benchmarks on 2 CPU and 4 CPU Opteron machines. Mostly we tested insertion times with many simultaneous connections (hundreds). To our surprise we did not find significant differences.Each of our servers (2 and 4 CPU machine) has one 1Gbit network connection. I have feelings that the machines are the network bounded, not the CPU bounded. Could it be true? How to verify it?
View Replies !
Mysqlcheck Performance
I launch the command line: "mysqlcheck --repair dbname" over a database with only one myisam table of more than 10.000.000 rows and 5 GB. The elapsed time to repair is about 1 hour! Is possible to increase speed?
View Replies !
Low Performance With Linux SMP
I use #mysql -V mysql Ver 12.21 Distrib 4.0.15, for suse linux (i686) with a SMP-kernel When I used this version with a kernel not SMP mysql was very very fast but now with the SMP-kernel the mysql is very very slow. I takes several minutes to make an INSERT on a table with near 50.000 Records Is there any problem with mysql an SMP? is a problem of the version I have installed?
View Replies !
|
TRACKER
