Best Text Search For Many Gigabyte Database?
I have a so big DB, It's about 10-100GB of text.
Now I want to create a searching function but It take too long when I
use LIKE from MySQL
Anyone know the best sollution?
View Complete Forum Thread with Replies
See Related Forum Messages: Follow the Links Below to View Complete Thread
Database Search -text Database...
Can anybody tell me how to search a flat file database delimited by |. I have exploded the data and managed to get it into a table but want to restrict the final table contents by a search. The table code at the moment is below - I'd be using a htm file to submit the data. and hopefully be able to search by partial searches eg big would return big and bigfoot. Code:
Smart Text Search - How To ?
I've got a list of names in a mysql table: example: Mike John Peter .... (100,000) entries How do I perform a search in this table, so that it is tolerant to mild spelling errors ? Am I looking for a phoenetic search algo ? Or is there a simpler algorithm to do this ? Or perhaps a PHP library for this ? For example if someone searched for "Jon", he should find "John" from the table.
Search A Text For Linebreaks
Got some text that contain url`s. Whenever there is an occurence of the string "http://" its going to read what comes after it - until next whitespace and make a link of it. But theres a problem when the link is at the end of a sentence, then it doesn`t treat the linebreak as a whitespace and the first word on the next line is treated as a part of the link. Is there any way I can search for linebreaks in a text/string?
Text Search Question
I'm working on a new project with php & mysql. I anticipate that one of the tables may grow above 40 GB in a short time. The main content is text. Now what tools do I need to implement a text search in this big text database ? I'd like the search to be quite advanced, supporting boolean operations and metaphones and so on. What should I be surveying now ?
Search A Text File
I have a text file that each line is an array, I think. I looks like this name | id | # | # | # | total name | id | # | # | # | total and so on... I would like to seach the file by "id" and only display the line with the particular "id". I also think that the bit size of each line will change from line to line, so I don't think that I can put in a static number to search to the end of. I thought about using fseek(), is that the best way to go.
Search A String Of Text...
I would like to search a string of text for "x" if "x" is found then it echo's something, else it echo's something else... Anyways I'm kind of lost as to the best way to search a string of text (for example "1,2,3,4,5,6,7,8,9,10,11,12") and find lets say "7,"...
Search Through A String For Various Text
My users enter text into a database field in the form of: this file {link:afile.pdf} is big and this file {link:another.doc} is small. I have no idea how many times they may enter {link:xxx} into a given field. The code we worked out (see other post below) fabulously translates the text into a hyperlink. The problem is that no matter how often the {link:xxx} is in a given field, I only ever get the first file returned; i.e: This file afile.pdf is big and this file afile.pdf is small. But what I should get is, obviously: This file afile.pdf is big and this file another.doc is small. I need to use some kind of looping function, or a foreach type function. Does anyone have any clues? Code:
Search And Replace In Text File
I have a text file with 30 or so lines. Each line has this format: sometext somevalue. They are separated by a single space. I want to search in the file for "sometext" and replace "somevalue" with a particular string.
Full Text Search ( Mysql )
I was searching the sql man , for some good ways to search a mysql db. so i found that Full text search the best way is to go becease LIKE just gives to mutch output.PHP Code:
Eregi Search And Replace For Some Text
I'm trying to do a search and replace for some text. For example, I have a word that is 'history', I want to replace every occurence of 'history' with: <a href="history.html">history</a> Now, that's easy, I can do that - however, as 'history' could also be 'History', I want to ensure that the format of 'history' stays the same. For example: $strRep = 'history' $str = eregi_replace($strRep, '<a href="history.html">'.$strRep.'</a>, $my_other_string); That's fine - however if history is capitalised (History), then I have a problem: $str = eregi_replace($strRep, '<a href="history.html">'.$strRep.'</a>, $my_other_string); because the above example would lowercase History to history.
Open, Search, And Process Other (text) Files
How could I use php to open, read, and perform operations on a file? Say I wanted to open somefile.htm and search for instances of <p class="one">Foo</p>, strip the tags, then put 'Foo' in a variable called $foo... Is this doable?
Highlighting Search Text, But Not Inside Tags
I've looked at some examples of text highlighting in this forum, and they all work fairly well. But I have a problem with my html tags. I have a site where the main body text is kept in a database text field. This text can contain some simple html. In particular urls. These urls get completely messed up with the css-tags that i insert to highlight the search words. Is there a simple regex expression that can make my script skip anything between < and >? I am currently using the example from this thread:
Dictionary Search Definition Replace Of Text
I'm trying to search a weather report for matches to the keys in my associative dictionary array and replace them with the $term => $definition. following is my code. again, any advise appreciated. REF: full php source PHP Code:
Full Text Search In PDF And Word Files ?
I need to perform full text searches on a batch of PDF and Word files. What is the best way to go? After some research, I'm thinking of extracting the plain text from the files with "pdftotext" and "catdoc", hamonizing the various possible encodings to UTF-8, storing the text in a MySQL database, and then using the full text search capabilities of MySQL. Do you think that would work well? I am told that the files are mostly text and won't be longer than 30 pages.
Full Text Search With Pagination Class
I've used the Full Text search example from this site and added the pagination class from http://www.goodphptutorials.com/article/show/simple-php-mysql-pagination/1 but I'm stumbling where it actually breaks the page up into the prescribed number of page. Right now I have script set for 5 results per page and while I can get the total number of records in the search result set, I can't get the results to span over say 3 pages if there's 15 results. Code:
Search From Database
IM looking for a better way to get results from search? IM currently using mysql_fetch_array and don't want my search to be in order. So if someone search for a word mysql_fetch_array will search in order of witch one was added to the database or from last. What I would like to have happen is more random or wildcard search. Is there a better way?
How To Search In A Database
I have a table called "lista" into my mysql database.This table has 5 text columns : "TRANSLATEDTITLE" "ORIGINALTITLE" "ACTORS" "DIRECTOR" "YEAR" , and 1 image column called "COVER". I want to create a page with a form to search entries into this table, and anotger php page with the search results shown inside, everyone with a link that opens a pop window (ex. 400X320).
How To Search In A Database
I have a table called "lista" into my mysql database.This table has 5 text columns : "TRANSLATEDTITLE" "ORIGINALTITLE" "ACTORS" "DIRECTOR" "YEAR" , and 1 image column called "COVER". I want to create a page with a form to search entries into this table (ex. with Title, Actors, Director, Year), and anotger php page with the search results shown inside, everyone with a link that opens a pop window (ex. 400X320). Who does give me an hand ??? I've seen the Codewalkers solution, but it searchs with another metod of searching (a selectable string with different category of searching). I need a simple search script with these two pages...
Search A Database
how do i seach through a specific column of data within a database for the highest value?
How To Search Without Database
i am developing a site and everything on the site is static meaning no database. now, they wanted to add a search tool bar thats is only meant to search by keywords my problem is i dont know how to do it because i have done it before.
Search Without SQL Database
Anyone able to point me in the direction of a no database php search tutorial or throw up an example. Basically, I need to search for non HTML elements pages within a directory.
Help On A Multiple Keyword Search On A TEXT Field Type
Working on a php/mysql driven site which requires me to have a search facility based on multiple keywords. I read the tutorial "quick and dirty search engine". What i want to know is : 1. Is there any SQL query i could perform which could to almost the same thing ( i dont really need the accuracy ) ? 2. Any other methods of achieving it?
Search For A Specific Word Inside A Text File?
Basically what i'm trying to do is when a user inputs login/password information at a login page, I want PHP to search inside verify.txt and if it finds the login/password combination then allows the user to proceed. Is this possible? And if so, which functions would I use to get the job done? Also, how can I save the login name so that it can be passed to/included in a url?
Search Database - MATCH() AGAINST() Or LIKE
I have a fairly basic database that holds articles that visitors of my web site can search based on content of the articles and the author. Simplified the tables look like this... articles +-----------------+--------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +-----------------+--------------+------+-----+---------+-------+ | doc_id | varchar(8) | | PRI | | | | title | varchar(255) | YES | MUL | NULL | | | date | date | YES | | NULL | | | source | text | YES | | NULL | | | abstract | text | YES | MUL | NULL | | | body | mediumtext | YES | | NULL | | +-----------------+--------------+------+-----+---------+-------+ authors +---------------+--------------+------+-----+---------+----------------+ | Field | Type | Null | Key | Default | Extra | +---------------+--------------+------+-----+---------+----------------+ | author_id | int(11) | | PRI | NULL | auto_increment | | doc_id | varchar(8) | YES | | NULL | | | author | varchar(100) | YES | MUL | NULL | | +---------------+--------------+------+-----+---------+----------------+ The Authors name is kept in the form of first initial, middle initial (if known) and last name (ie J. Bloggs). Basically users have two fields they can complete for searching. One is the keywords (ie words in the body of the article) and the other is for the author of the article. Currently I perform a full text search to find authors that match the author they enter but I'm not sure this is the best method as authors with short last names (ie. Li or Cho) can't be found. This is the code I am using to generate a query to search the database. PHP Code:
Search Item In Database
I made a function that does a case insensitive search. This works fine. But i'd like to improve that script so that it also returns items from the database that are partly similar to the search string. this is the script atm:
Database Search Engine
how do i make php script for searching in mysql database? i want to make a student search.. the field is student_number, name, address, and faculty. what would the php script look like?
How To Search More Than One Table In A Database
Say I have a user input a search string and they click search. What would my mySQL string look like if i wanted it to search TABLE:'set_01' and TABLE:'set_02' at the same time for the same information?
Some Database Search Tips
I have this database where I get like thousand of searches every day. But I think it is running a bit slow. Do you have any tips on how to improve it and keeping it flow.
Database Search Problems
I have managed to connect to the database fine and I think I understand that I want to post the information, and I am posting it to the page that I am currently on, as I want the results to be displayed below the search form. There should only be one result no matter what search in entered. I am getting some results, but they are inconsistent. Postcode seems to work but nothing else, the other 2 input fields always return the third record in the database no matter what value is entered. Code:
Database Search Results
i have a paid to read website,im hosting it off my own pc..as a member you can search offers and complete them, heres where i need some help..when a member clicks submit and the offer goes to an offerdone table i need the offer to hide from them so they cant do it again,also that same offer needs to hide in the search results as well...is there somthing like onclick hide and for the results something like if duplicate hide from search or if offer found in this table hide from results... there are 2 tables one will all my offers the other with offer they have done,also this needs to apply for each member account.i am new to all this and have made it this far to have my site up and going but im looking for those extra catchs i could really use,so that a member cant repeat an offer they have already done,
Full Text File Search With Indexing Service On Windows
Here's a short tutorial on how to the OLE-DB extension to access Windows Indexing Service. Impress your office-mates with a powerful full-text search feature on your intranet. It's easier than you think. First, download and install the extension (http://sourceforge.net/project/show...kage_id=198554). Simply unzip the file and copy the correct version of php_oledb.dll into the PHP extensions folder. Then add the line extension=php_oledb.dll in php.ini and restart your web server. Now, if Indexing Service isn't running on your computer, turn it on. Go to Control Panel Administrative Tools Services and configure Indexing Service to start automatically. You can also ask the little dog in the search window to do it for you if you're using Windows XP. You will need to wait a while for Windows to build the initial index. It could take a couple hours. Once the extension is installed and the index is ready, you can start coding. To connect to Indexing Service, you use the oledb_open function: $link = oledb_open("Provider=MSIDXS"); You then call oledb_query with a SQL statement. Let us start with something simple: We'll look for all files on the computer containing the word "love": .....
Suggestions For A Database Search Engine?
The site that I am currently building has a products catalogue stored in a database spread over three tables: products (holds all the product information, brand, model, description, price etc...) categories (Holds all of the categories and sub-categories) products_categories (links products and categories tables). Right what I want to do is implement a search engine that will search through the products table for the keywords inputted by the user and, displays a list of all the products / pages with the relevent information. I am rather new to PHP and MySQL and I'm not sure which way to tackle this, any suggestions would be appreciated. If you need any more info on the the tables and whatever please say.
Fastest Way To Do Database / Array Search
I am trying to write a script that allows users to search through a database of names. But rather than give a search string and then return all those that match, I would like it to search each time the user types a new letter in the text box. That's badly explained - this example might help! The user types "r" and I displays all the names with r in them. The user continues and types "i" so the textbox shows "ri" and displayed are all the results with "ri" in them. I as doing currently this by re-querying the SQL database where the names are hosted but it takes too long to query, return and refresh the page. What would be a better way of storing this database info to allow a faster search? Maybe load it into a PHP array first (stored in another file) and search within that? (Can you do 'clever' searches in PHP?) Or into a javascript array? The list could be upwards of 1000 names. The databse shouldn't change that often so I could have a script that updates the array every 24 hours or every time a new name is added.
Search Tags In Mysql Database
I'm creating a website that lists snippits of online articles. As tagging these articles will be very important, I've created a database structure as below. I'm now implementing searching of these tags with the hope to sort potentially 1000's of matches by relevance on tags and on date (the newer the better). Before launching head first into this, can anyone point me in the way of a good tutorial specifically on tag searches. (I can find plenty of fulltext sytle search information, but not so much on tagging). Code:
MySQL 5.0, FULL-TEXT Indexing And Search Arabic Data, Unicode
MySQL 4.0, FULL-TEXT Indexing and Search Arabic Data, Unicode [This version has a couple subtle edits from the orginial I posted on mailing.database.myodbc - I'm cross posting here on this topic/subject related newsgroup] I was wondering if anybody has experienced the same issues challenges I'm experiencing I'll describe shortly. Once resolved some fascinating and powerful multi-lingual apps incorporating non-English/latin character sets can be realized by many developers. I have a Unicode utf8 English - Arabic - Hebrew - Greek (and several other languages) database in Microsoft Excel. I KNOW that it is Unicode utf8 data because MySQL tells me it recognizes the encoding as such but not in the context I want. Allow me to explain ... I can search the Unicode utf8 encoding with no problem in Excel. While in Excel I highlight a complete word or a partial string of an Arabic word copy it to the clipboard (i.e. memory). I then do a find and the process is the same successful result as if it was an English string. MySQL 5.0 is supposed to handle Unicode utf8 I created a MySQL database I named: languages CREATE DATABASE languages ; and I implemented the following command on a MySQL command prompt: ALTER DATABASE languages DEFAULT CHARACTER SET utf8; No problem (so far) MySQL seemingly recognized utf8 and accepted it. My understanding is with the ALTER command the tables I create against languages will be utf8. I now created a table I named mainlang which denotes it will be the main table for my languages. mysql>CREATE TABLE mainlang ->( ->langNumID varchar(30), ->colB varchar(30), ->colC varchar(30), ->primary key (langNumID, colB) ->); Again so far no problem: Table successfully created. My third column 'colC' is where the Unicode data will be stored. I now attempt to import the database from my Excel file into my MySQL database as follows: mysql>load data infile 'c:arabicdictionary.csv' ->into table mainlang ->fields terminated by ',' ->lines terminated by ' ' ->(langNumID, colB, colC); ERROR 1406 (22001): Data too long for 'colC' at row 1 So what to do? I did a search and found other people seemingly had the same problem and someone suggested: ALTER DATABASE languages DEFAULT CHARACTER SET cp1250; I dropped mainlang, recreated it, redid the load and Lo and behold ... it seemed to work. No Data too long error occurred and when I did the following query: mysql>select langNumID, colB, colC ->from mainlang ->where colB = ��' I see colA have a correct numeric value, colB a correct numeric value (4994) and for colC a string of unintelligible characters with diacritical marks, oomlats etc. which I know is the cp1250 encoding interpretation of the Unicode utf8 data which is similarly unintelligible in its own regard. Now what I try is: do a copy of the obscure colC cp1250 character string into the clipboard/memory and then do the following tweak on the original select statement to see if I can search on the (now) cp1250 character string: mysql>select langNumID, colB, colC ->from mainlang ->where colc = 'paste of the cp1250 character string' The computer would not allow a paste unless I pressed the escape key. On initiating this select command I got an empty set (no match) My questions are: Has anyone been successful creating a Unicode utf8 MySQL database that accepts Arabic? If yes, how did you get around or not encounter the Data too long issue? Have you tried the cp1250 (or cp1251 - same mechanics same results) work around as I have? Are you able to search the cp1250 character string (my colC)? If yes, how did you successfully manage to do it? Lastly, if I take the cp1250 encoded string and paste it into Excel ... I can string search the cp1250 encoding with no problem. Also, here's how I know my Unicode utf-8 data is correct apart from my own manual cross-referencing and being recognized by MySQL in some respect: When I copy the Unicode utf8 encoding and try to paste it into the select command to see what would happen I get the following error: ERROR 1257 (HY000): Illegal mix of collations (cp1250_general_ci, IMPLICIT) and (utf8_general_ci, COERCIBLE) for operation '=' So what I have here is a situation where MySQL is recognizing Unicode utf8 encoding but not from the respect of packing a table! Go Figure ...
Give User A Choice Of Fields To Search With Full Text/Boolean?
I want to give the user the option of searching a combination of the fields $Topic, $Subtopic, $Theswords in Boolean/Full Text. The problem is if I make any of the fields empty prior to the query, I get a MySql syntax error message. The user has the option of selecting fields via checkboxes, but when the field is not selected the user will get the error message because the variable is empty in the query statemtnt. Are there any code suggestions for avoiding the error message when a field is not selected by the user to be searched? Code:
Database Driven Pages On Search Engines
I know this has been discussed at length in these forums from the searches that I've done, but I have a new question of sorts. Basically, I found several solutions to the problem of using PHP/MySQL database driven pages and getting them listed on search engines. One method that I've chosen entails listing the variables in between backslashes after the name of the document, like this http://www.something.com/index.php/test/1234 After a few problems, I've been able to get all of the pages in the first group (I just added the 2nd and 3rd group today) to list perfectly on Google. I'm psyched about this working on my entire database of 10,000+ products, but I'm worried that I may be overlooking something. If possible, please check out that link and if anyone knows of any shortcomings or pitfalls with using this method, please let me know. Thanks a ton.
Creating A Search Class That Searches Any Database
I've been requested by a client to implement a search engine on it's website. I told him that he does not need it, but he thundered on and well...Who cares, he' s gonna pay for it. I've made search methods for other sites, but I would like to implement a portable one. A Search Class that could work for any DataBase and with ease of configuration. Maybe de constructor could receive a database name in which to search. And the function search, could receive an array of tables in which to search on and a string to search for on those tables. The question is...how would u implement the way that the class returns results? Nowadays any pro site has the MVC controller. That means that almost all tables in the database have its corresponding model. Table clientes correspond to the Client Model. How would you implement the results return?
Text Not Right From Database
I have a Profile section in my users registration, but if you enter a few lines and a few 'enter spaces' like <BR>, then the <BR> spaces do not appear and the text is continuous! my profile section is just 'text' in the database..should i change this to blob or varchar.? I also need the same fix for my news part! which is over 255 chars!
Database Row To Text File
I need to get the contents of a MySQL database field into a text file, and write the file to the server. I can not seem to get any filesystem functions to work, because it tells me I do not have permission to do anything. I am on virtual hosting, so it seems easier for me to try to FTP files to my own server. Perhaps you guys can give me some pointers, this is kind of the idea I am running on:
Plain Text Database
i'm really a newbie to php but not OOP. i'm designing a database to hold simple text messages to display in a page called, "News". The client doesn't want a sql database so I suggested a plain text database. I have it working but when I pull the data (fopen) it all comes back as one line. It's set up as a simple form passing 2 variable, $title and $comments. They both write (fwrite) just fine to the .txt file but upon retreiving them (fopen) it's all one line. Since I can't pass formated text to a .txt file is there a different way? As a newbie I haven't come across a solution yet. The client wants this soon so I'm asking here due to the timeline. Given a few more weeks I'm sure I'd stumble across it in some text.
Text File Instead Of Database?
Does anyone know a script that I can use to manage members of my website (preferably with a control panel-type interface) but without using a database? I read on hotscripts.com of one that uses text files instead, but the link says that the site is no longer offering that file. If someone could help me it would be great, as I know nothing about scripting. By control panel, I mean a page that I can go to on my site (not open to the public) that I can use to Admin the users. I would like it to go something like this: (this is what was offered at hotscripts.com) New Members Username: New Members Password: New Members Name: New Members E-Mail Address: And optionally (I don't really need it, but if its possible, hey why not) a place for me to put in an admin username and pass to get into the page mentioned above. I would also like to be able to implement this script on my main page (see here for what I mean.
Preparing Text For Insertion Into Database
I insert text into a MySQL Database using a TEXT field. PHP syntax is: $query = "INSERT INTO data (text_field) VALUES('$text_field')"; The problem is, when the user has an apostroph (like "friggin' cool") in the text, MySQL misinterprets it, and the query is invalid. Is there a PHP function that formats the that input so it can be entered into the database?
Storing Article Text In A Database
I have a website with an increasing amount of articles and news reports and so I am thinking of moving away from storing each article as a seperate page to having a single page and storing articles in a databasewhich are retrieved using a GET parameter. I see the advantage to me in using this approach as being making use of MySQL's fulltext search capability and less work needed when updating the design of the page. I'm not sure of a few things though: 1. The exact benefits of fulltext. Can someone provide a brief explanation or a good website 2. I imagine that storing just the plain text of the article in the database will result in a single block paragraph when retrieved into the page. Do I need to include the HTML markup with the text in the database or is there a different approach? 3. Will the HTML markup in the article interfere with the ability to search for words i.e. if I have a word "recipes" in an article that is surrounded tags like <b>recipes</b>, will a search on this article pick out the word from in between the tags?
Load Into Database From The Text File
i'm having difficulties here to load a text file to my database... let's say if i've a table named "train_info", and the field for that particular table is user-id, training_hours, training_date. let's say i've a text file that looks like this : " 123434 12 2002-08-03 545322 10 2002-08-03 533434 12 2002-08-03 654333 12 2002-08-03 535434 12 2002-08-03 " Is it possible for me to read such text file to be inserted into my "train_info" table?
|
TRACKER
