logo.gif The source for online courses
in statistics
 ÖÐÎÄ Course Login
Home > Resources > Discussion Boards
statistics.com
statistics.com
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

what can I do when my logistic reg model has poor calibratio

 
Post new topic   Reply to topic    statistics.com Forum Index -> Statistical Methods
View previous topic :: View next topic  
Author Message
sharon19



Joined: 09 Apr 2010
Posts: 21

PostPosted: Mon Jun 21, 2010 1:41 pm    Post subject: what can I do when my logistic reg model has poor calibratio Reply with quote

For medicine and AUC >700 is considered good, so I am okay. I have AUC of 760
BUT my H-L test is p=0
I have read about using clustering as a way to increase calibration, but the paper was complex. I read somewhere that the calibration does not matter. I want to write up a paper so I do not know what to do.

My equation is simple
Resuscitation Code Status = AGE + GENDER + SAPS SCORE

Besides making AGE binary, which I have had trouble doing, I do not know what to do.

Ultimately I am interested in the model to derive Odds Ratios to determine ageism or gender bias

Many thanks.

Sharon Smile
Confused

Oh, and though I do not think this will work, if AGE (continuous) does not have an interaction term, do I have to repeat the interaction study if I change AGE to binary?
Back to top
View user's profile Send private message
alethephant



Joined: 06 Sep 2006
Posts: 200
Location: Virginia Beach

PostPosted: Mon Jun 21, 2010 3:32 pm    Post subject: What can I do when my logistic reg model Reply with quote

1. What is "AUC"? Is this "Area Under Curve" in a ROC plot? If so, does "760" mean 0.76?

2. What is "H-L"? Is this the Hosmer-Lemeshow test? If so, you need to describe more about how you carried it out and how many classes you used and how many data you have.

3. AGE is a complicated variable, and you don't describe what range and ages you have, and how you expect them to relate to your RCS response.

4. What are your "clusters"? Are these institutions you sampled at?

Etc.
Back to top
View user's profile Send private message
sharon19



Joined: 09 Apr 2010
Posts: 21

PostPosted: Mon Jun 21, 2010 3:37 pm    Post subject: Reply with quote

Yes to numbers one and two, sorry for my abbreviations.
Age ranges from 15 - 100 exactly (due to de-identification). The mean is around 65 years
I have approximately 17,000 subjects.
I do not have any clusters, it was just a thought. All the data comes from one hospital, but it does come from different ICUs (about 7 of them)


Thank you
Back to top
View user's profile Send private message
alethephant



Joined: 06 Sep 2006
Posts: 200
Location: Virginia Beach

PostPosted: Tue Jun 22, 2010 9:41 am    Post subject: What can I do when my logistic reg model Reply with quote

The 7 ICUs are probably clusters (correlation among subjects in same ICU).

You have a wide range of AGE, so if you are going to categorize it, you should use more than 2 categories. Perhaps 4 or more to capture the association.

As to your question of "calibration", which I presume means "goodness of fit" or "stability of fit", validation is a good idea if you have a lot of data.
Back to top
View user's profile Send private message
sharon19



Joined: 09 Apr 2010
Posts: 21

PostPosted: Tue Jun 22, 2010 2:38 pm    Post subject: Reply with quote

what do you mean by validation? Yes I mean goodness of fit.
Thank you,
Sharon Smile
Back to top
View user's profile Send private message
alethephant



Joined: 06 Sep 2006
Posts: 200
Location: Virginia Beach

PostPosted: Tue Jun 22, 2010 6:45 pm    Post subject: What can I do when my logistic reg model Reply with quote

Validation usually means verifying stability of a model by holding out portions of data and seeing if the model remains essentially the same when fit on different portions.

This can be a dichotomy, such as 70% for training and 30% for validation, or a trichotomy of 50% for training, 30% for design level validation and 20% for final independent validation of the selected model.

Or it can be a more systematic approach, such as jackknifing, bootstrapping or other techniques with a sampling component.
Back to top
View user's profile Send private message
sharon19



Joined: 09 Apr 2010
Posts: 21

PostPosted: Tue Jun 22, 2010 8:42 pm    Post subject: Reply with quote

thank you very much. I have used validation for classification, but I did not realize it meant the same thing here. Thank you
Back to top
View user's profile Send private message
LeeRain



Joined: 25 Jun 2010
Posts: 3

PostPosted: Fri Jun 25, 2010 2:38 am    Post subject: Reply with quote

You have a wide range of AGE, so if you are going to categorize it, you should use more than 2 categories. Perhaps 4 or more to capture the association. Laughing
_________________
import trade lead
brass disks
Back to top
View user's profile Send private message
Sam13



Joined: 31 Jul 2010
Posts: 1

PostPosted: Sat Jul 31, 2010 1:17 am    Post subject: Reply with quote

Thank you everybody...
Igot many useful information from you.... Laughing Rolling Eyes
_________________
best way to get rid of acne
Back to top
View user's profile Send private message
zihad36



Joined: 18 Aug 2010
Posts: 1

PostPosted: Wed Aug 18, 2010 1:30 pm    Post subject: Reply with quote

I have approximately 17,000 subjects.
I do not have any clusters, it was just a thought. All the data comes from one hospital, but it does come from different ICUs (about 7 of them)



online bingo
Back to top
View user's profile Send private message
raat475



Joined: 28 Aug 2010
Posts: 1

PostPosted: Sat Aug 28, 2010 4:02 pm    Post subject: Reply with quote

I read somewhere that the calibration does not matter. I want to write up a paper so I do not know what to do.

My equation is simple
Resuscitation Code Status = AGE + GENDER + SAPS SCORE

Besides making AGE binary, which I have had trouble doing, I do not know what to do.



retouching
Back to top
View user's profile Send private message
zia111



Joined: 04 Sep 2010
Posts: 1

PostPosted: Sat Sep 04, 2010 6:30 pm    Post subject: Reply with quote

I have read about using clustering as a way to increase calibration, but the paper was complex. I read somewhere that the calibration does not matter. I want to write up a paper so I do not know what to do.




no deposit bingo
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    statistics.com Forum Index -> Statistical Methods All times are GMT - 5 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group