Retrive Score In Logistic Regression (Microsoft Neural Network Viewer - SQL Server 2005)
Feb 19, 2008
Hi!
I bought the book €œData Mining with SQL Server 2005€?, but I can€™t find the solution to a problem I have.
I want to retrieve from C# the logistic regression Attribute Value (AV) Scores for the Logistic Regression Algorithm. I can see the Scores from the Microsoft Logistic Regression Viewer (the same of Neural Network Viewer), but I cannot retrieve them via DMX, OLEDB or similar.
Otherwise, is there a formula that I can use to compute that score from the coefficient, support, or probability values of the Attribute Value pair (I can read this values from DMX)?
I can access to them via DMX:
NODE_DISTRIBUTION -> SUPPORT and PROBABILITY ATTRIBUTE_VALUE...
with a query like
SELECT FLATTENED (SELECT ATTRIBUTE_NAME, ATTRIBUTE_VALUE FROM NODE_DISTRIBUTION WHERE VALUETYPE = ... ) FROM [MyModel].CONTENT WHERE NODE_TYPE ....
This question is regarding the LogRegHelper - "A scorecard for Logistic Regression models" example in sqlserverdatamining Tips and Tricks page. I launched TestLogReg (Analysis Services Database associated with the project) and ran Logistic Regression over that. While the LogReg shows the highest score for IQ (107 - 121), a score of 558, the Logistic Regression shows that Parent Encouragement has the highest score for the case College Plans = 'Plans to Attend'. Can someone verify this and clarify?
I have a few other questions with LR
- In SQL Server 2005 LR Mining Model Viewer "favors" chart, what algorithm is used for generating Scores?
- Can I use this score as a feature selector? Higher score => stronger predictor (input)
- Is the coefficient weight algorithm used in LogReg wrong ?
I have read the threads regarding the Neural Network Viewer and I think I have a similar problem. I do have Service Pack 2 installed and I'm running the x64 version of SQL 2005.
I'm building a model from a single relational CASE table. Granted the table has many columns summarized at the customer level but there are it is well formed and has no NULL values (plenty of zero or blank values though). The only time I can get the NN Viewer to work is when I accept the attribute recommendations. It seems once I stray from these recommendation, even if there is still correlation with an attribute, I cannot view the model using the NN Viewer. My latest error message says:
"The provider could not determine the String value. For example, the row was just created, the default for the String column was not available, and the consumer had not yet set a new String value."
I get this message even when all input attributes are Continuous so I'm not sure what String column it is referring to.
Any help is greatly appreciated. I'm in a time crunch and I have sold the client on SQL Server 2005 capabilities. It's a bit embarrasing if I can't get this resolved.
-- Steve
P.S.: I don't recall having any issues with the NN Viewer prior to Service Pack 2 (although others have). Have you done regression testing to test this issue?
I am confused about the value of Probability of Value 1 or 2 (on a particular attribute value) in Neural Network viewer. E.g. the value of Probability of value 1 is actually very low (the same to the value of Probability of value 2), but why the bar which shows the strength of the probability of these two values are still so strong even stronger than other values of probability of value 1 or 2 based on other attribute values which have a much higher probability of value 1 or 2?
And how does the algorithm calculate the Probability of attribute value in nerual network by the way?
Hope my question is clear.
I am looking forward to hearing from you shortly and thanks a lot in advance.
I've successfully created and processed a very simple neural network mining model (defined against a cube). However, when I go to the model viewer in BI studio, it displays the following error:
"Execution of the managed stored procedure GetAttributeScores failed with the following error: Exception has been thrown by the target of an invocation.Input string was not in a correct format.."
Any ideas about what's going wrong? This is with SQL Server 2005 SP1.
I'm working with Analysis sevices 2005 developer edition. Looking through the documentation i becomes apperent that the NN algorithm takes 255 input attributes by default. This can be changed to any integer value, OK....
My problem is that I want to feed the network with 40000 input variables. In order to do so, I will have to do a select:
SELECT fld1, fld2, ...... fld39999, fld40000
FROM tblSometable
However, this is not possible, as the books online describes it is only possible to return 4096 columns from a select statement.
Question : How do I populate a NN in AS2005, with nmore than 4096 inputs ?!
Say that I have 100,000 attributes/feature selections for my SQL Server Neural Network Algorithm.
Customer Attr1 Attr2 Attr3 ..... Atr100000 ============================== Jack 1 0 1 ..... 1 Sam 0 1 1 ...... 0 Mary 1 1 0 ...... 1
Knowing the fact I can't fit those info on a table and SQL Server's Neural Network does not support table prediction . What's an alternative to use Neural Network in SQL Server 2005 to solve my problem?
Microsoft.AnalysisServices.Viewers.DLL dll in VS 2005 but sometimes it gives an error.
"Code generatio for property 'ConnecitonManager'" failed. Error was:'Property accesor 'ConnectionManager' on object 'AssosiactionViewer1' threw the following exception:'Object referance not set to instance of an object"
Is there anyone here who use "Microsoft SQL Server 2005 Datamining Viewer Controls" in SQLServer2005 FeaturePack ? http://www.microsoft.com/downloads/details.aspx?FamilyID=50b97994-8453-4998-8226-fa42ec403d17&DisplayLang=en
i am using VS2005 Version 8.0.50727.762 (SP.050727-7600) and SQL Server 2005 SP2
I downloaeded "Microsoft SQL Server 2005 Datamining Viewer Controls" feature pack, and tried to run the dataminingviewerclient.exe.
When i view my local time series data mining model, It works well for single series model, but when i load a Microsoft time series model containing multiple data series, an unhandled exception occurs.
I post the error message below:
See the end of this message for details on invoking just-in-time (JIT) debugging instead of this dialog box.
************** Exception Text ************** System.NullReferenceException: Object reference not set to an instance of an object. at Microsoft.AnalysisServices.Viewers.TimeSeriesViewer.PreprocessChartArrays(ArrayList rgPredicted, ArrayList rgActual, ArrayList rgDeviations) at Microsoft.AnalysisServices.Viewers.TimeSeriesViewer.LoadChartSpace(MiningModel model) at Microsoft.AnalysisServices.Viewers.TSCheckedListViewCombo.HandleChangesCommitted(Object sender, EventArgs ea) at Microsoft.DataWarehouse.Controls.BaseControlComboBox.OnCommittedChanges(EventArgs e) at Microsoft.DataWarehouse.Controls.BaseControlComboBox.SetSelection(String str, Object obj) at Microsoft.AnalysisServices.Viewers.TimeSeriesViewer.InitializeCurrentlySelectedPage(Object context) at Microsoft.AnalysisServices.Viewers.TreeViewer.viewerTabControl_SelectedIndexChanged(Object sender, EventArgs e) at System.Windows.Forms.TabControl.OnSelectedIndexChanged(EventArgs e) at System.Windows.Forms.TabControl.WmSelChange() at System.Windows.Forms.TabControl.WndProc(Message& m) at System.Windows.Forms.Control.ControlNativeWindow.OnMessage(Message& m) at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m) at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)
************** JIT Debugging ************** To enable just-in-time (JIT) debugging, the .config file for this application or computer (machine.config) must have the jitDebugging value set in the system.windows.forms section. The application must also be compiled with debugging enabled.
When JIT debugging is enabled, any unhandled exception will be sent to the JIT debugger registered on the computer rather than be handled by this dialog box.
***** end of error message***
Can someone help to figure out what went wrong? thx.
I have a mining structure that I am using to perform a text-mining classification task. The mining structure contains three models: a decision tree, a naive bayes and a neural network.
Both the decision tree and the naive bayes models process without any problems, but I am having significant difficulties with the neural network model.
Initially when I processed the model, processing would fail altogether with the following error message:
"Memory error: Allocation failure : Not enough storage is available to process this command"
This was remedied by taking the steps prescribed in http://support.microsoft.com/kb/917885 (I upgraded to SQL 2005 SP1 and applied all available hotfixes listed in http://support.microsoft.com/kb/918222/). This got me to the point where the model (seemingly) processed correctly after restricting the MAXIMUM_INPUT_ATTRIBUTES to a relatively low number. So after processing, I went to try and browse the neural network model and view the lift chart...
<error>
"Execution of the managed stored procedure GetAttributeScores failed with the following error: Exception has been thrown by the target of an invocation.Input string was not in a correct format.."
</error>
(see http://forums.microsoft.com/TechNet/ShowPost.aspx?PostID=935340&SiteID=17)
Also when I would attempt to view the lift chart and the classification matrix the queries would time out with the following error message:
<error>
XML for Analysis parser: The XML for Analysis request timed out before it was completed.
Execution of the managed stored procedure GenerateLiftTableUsingDatasource failed with the following error: Exception has been thrown by the target of an invocation.Microsoft::AnalysisServices::AdomdServer::AdomdException.
</error>
Now, my poking around on Technet lead me to believe that this issue could finally be resolved by uprading to the CTP release of SQL server 2005 SP2. Now I am still encountering problems. When I go to browse the model in the Neural Network Viewer, I see the correct drop down menus to select attributes and attribute values in the "Input" and "Output" panes but I see no data displayed in the "Variables" pane at the bottom.
Interestingly, while I cannot view the model contents in the graphical viewer, the mining model contents viewer reveals model contents that look to be pretty normal for a trained neural network.
Attempts to view the lift chart time out with the error message:
<error>
XML for Analysis parser: The XML for Analysis request timed out before it was completed.
Execution of the managed stored procedure GenerateLiftTableUsingDatasource failed with the following error: Exception has been thrown by the target of an invocation.Microsoft::AnalysisServices::AdomdServer::AdomdException.
</error>
and when I run predictions against the trained NN model in the "Mining Model Prediction" pane it predicts the same value for every case in the testing set.
I have two problems while trying to train a neural network. My network have 10 continuous input ad 1 discrete output (3 states)
The parameters I chose are : -Hidden node ratio 10 -Holdout percentage 10
The others are default.
First,when i train it thanks to BI dev studio, the training is very fast (less than 5 seconds) and the results compared with the training set are bad (at least 30% of errors). Is there a way to improve the training (I don't care about the time required to train if it works)?
General data mining books talk about NN taking inputs which are between -1 and 1. Even Jamie's book says that's what it generally receives. I don't think this is a requirement for the Microsoft algorithm, but I wanted to ask if it was a best practice. If you're feeding it something like home values where 99% of homes are under $1 million you can use some normalization trick so that mansions don't skew the data. But if your data doesn't need such normalization, is there any need to normalize it to the -1 to 1 range?
Also, is the Microsoft algorithm sensitive to the relative size of different inputs? For instance, if InputA is home size (500-50,000 square feet) and InputB is months unoccupied (0-24 months), does that cause the Microsoft NN to weigh home size more heavily?
Hello , using MS Visual studio 2005 , I deployed sql table with NN algorithm , it successfuly deployed . But when I tabbed to "Mining Model Viewer" it gave me the following error :
The following system error occurred: Invalid procedure call or argument. Execution of the managed stored procedure GetAttributeScores failed with the following error: Exception has been thrown by the target of an invocation.Microsoft::AnalysisServices::AdomdServer::AdomdException.
With the number of threads it is difficult to know if this has been posted. If I use the Mining Content Viewer for Linear Regression, under Node Distribution, there are values given for Attribute Name, Attribute Value, Support, Probability, Variance, and Value Type. The output is similar to what Joris supplied in his thread about Predict Probability in Decision Trees. My questions:
1. How should these fields be interpreted?
2. With Linear Regression, is it possible to get the coefficient values and tests of significance (t-tests?), if they are not part of the output I have pointed to?
What can we tell from the lift value of the attribute value in Neural Network? (any so-called threshold of this lift value which identifies whether or not an attribute value is important or whatever to the selecte output attribute value?) I mean with lift value of a particular value, when we describe the characteristics of a segmentaion with Neural network, what can we actually describe it?
I just dont know what can we desribe this for the segmentation by neural network viewer?
I am looking forward to hearing from you shortly and thansk a lot.
I am in the process of training a Neural Networks, which could take significant iterations in the process of getting trained. While using other tools like I can visually see the convergence (in terms of error for the model). Is there a way to see any progress while training while using Analysis Server - Neural Network training? It would be useful to see the accuracy, interation number and timeout while in the process of training etc...
Is the standarization of the inputs done automatically when running the Microsoft Neural Network algorithm or I should be transforming the variables before running the algorithm?
2) Predicted Probabilities
How do I create a table with the actual predicted probabilities of the model for each observation? In the Mining Model Prediction tab the output would be either 0 or 1, my question is how can I obtain the actual value of the estimated probability?
I need some advise/help how to deploy a small sql server compact 3.5 database on networkshare where about 8 people have access to it(install de client ) and can modify the database with aprox 3000 records. I'm using visual basic 2008 and created a compact database and filled it with one table of an export of an access database done with data port console. And created a new windows forms application and saved the project. Publish is not the problem but how too set it up...... There is not much info to find on how to get a sql server 3.5 database running on a network share. Who can help ???
I m student and develop a website. i want to know how images /pics and video and audio files are store in Sql server2005 database. and ya i want to store whole file not just its address.
i using asp page like uploadImage.aspx in that i put UploadFiles control now i m confuesd to storing Images so plese help me and give code or some good easy trick at my level
And once store image or video and audio file in sql database then after how i used in my asp.net page. which control i wish to used for this files so plese this also slove it. give also code or any easy trick for fullfil my task..
I am using Reporting Services in the following way. The application implements a three tier Architecture viz Web Server, Application Server and Database Server. Reporting Services 2005 is installed on the Database Server.
The application uses ReportViewer Control which is part of an ASPX page deployed on the Web Server. For ReportViewer Control to fetch data from Reporting Services using RS API, we have to set ReportServerUrl and Report Path. In this scenario, the web server cannot communicate to the database server directly. The web server is in the DMZ and it can only communicate with the Application Server which is in a separate Network.
How can I use Report Viewer Control to fetch data from Reporting Services which is on the Database Server.
I am using MFC ODBC classes in my VC++ application for database accesss (on a SQL Server 2005 Database). After opening a Recordset I have to use the CRecordset::GetFieldValue( LPCTSTR lpszName, CDBVariant& varValue, short nFieldType = DEFAULT_FIELD_TYPE ) method of the CRecordset object to obtain field values. The above method works fine with ODBC drivers for Jet but when I use ODBC for SQL Server it throws the following exception.
State:S1002,Native:0,Origin:[Microsoft][ ODBC SQL Server Driver] Invalid Descriptor Index
I don't understand why the Microsoft SQL Server ODBC Driver is throwing this exception.
My application is running fine, as i can login to it and also able to view pages. But when i open Order management(having 3K records) its give me error,
General Network Error. Check your network documentation
I have also searched many articles and tried following solutions but nothing working
- connectiontimeout = 0, max pool size = 7500/100, pooling = false
- SSL disabling enforce security false as mentioned in microsoft kb article.
And there's nothing any issue with hardware/firewall as my application's login and other forms are working fine(which use same database with same connection string)
I'm new with SP v3 and the Reporting Services of SQL Server 2005.
I did a prototyp report and show it with "SQL Reporting Services Report"-WebPart, but I can't find the WebParts Report Viewer and Explorer in the WebPart Gallery.
Please, could somebody give me a clue where the problem could be, or how I could isolate the problem area.
I am update/inserting records threw a web form in vb.net. I need to insert 'NULL' into my microsoft sql server database. I am not talking about the below line of code where website is the name of my paramater. If i do that it will just place a blank into that field in the database. If i dont enter anything into that textbox I want it to to say NULL in that field. So if I go into the actual table in the SQL Server Management Studio and look at the website field of the recored I just added or updated and did not type anything into the web site textbox it needs to say NULL. I also tried the second line of code but that places a single quote in front and behind NULL. So the field will have the value 'NULL'. website.value = txtwebsite.text.tostirng if txtwebsite.text ="" then website.value ="NULL" end if The reason why I need the NULL there is because I bind the website filed to a hyperlink template in a gridview. The actual text of the hyperlink is bound to PAYER which is a name of a insurance payer but the navigateto is bound to the website field. If i do not enter any data into the website field and it stays as NULL, then when my gridview is loaded, payers that dont have a website will not be underlined and user wont have the option to click on them since there is no value for the navigateto. But if i use my form to update/add a payer and leave the website textbox blank in puts a blank into that field in my database and when it loads that new row into the gridview the PAYER is underlined and u can click on it but it will just take u to the web site is unavailable page. So is there anyway to actaull have NULL placed into a filed and not just blank space
