Creating meaningful cut-scores for Neuro-QOL measures of fatigue, physical functioning, and sleep disturbance using standard setting with patients and providers

Abstract

Purpose

To establish clinically relevant classifications of health outcome scores for four Neuro-QOL measures (lower extremity function, upper extremity function, fatigue, and sleep disturbance).

Methods

We employed a modified educational standard-setting methodology to identify cut-scores for symptom severity. Clinical vignettes were developed to represent graduated levels of symptom severity. A clinician panel and a panel of persons with multiple sclerosis (PwMS) were recruited, and, in separate, 1-day meetings, the panelists identified adjacent vignettes they judged to represent the threshold between two levels of severity for a given domain (e.g., threshold between a vignette that indicated “no problems” with sleep and the adjacent one that represented “mild problems” with sleep). Working independently, each panel (PwMS and clinicians) reached consensus on its recommended thresholds for each of the four targeted measures. Cut-scores were defined as the mean location for each pair of threshold vignettes.

Results

PwMS and clinician panels derived identical thresholds for severity levels of lower extremity function and sleep disturbance, but slightly different ones for upper extremity function and fatigue. In every case of divergence, PwMS set higher thresholds for more severe classifications of symptoms (by 0.5 SDs) than did clinicians.

Conclusions

The modified bookmarking method is effective for defining thresholds for symptom severity based on self-reported outcome scores and consensus judgments. Derived cut-scores and severity levels provide an interpretative context for Neuro-QOL scores. Future studies should explore whether these findings can be replicated and evaluate the validity of the classifications compared to external criteria.

Appendix: Clinical vignettes

(Note: T score locations are provided, but are not printed on participant copies)

Fatigue

Ms. Moore’s Fatigue (T score = 32.5)

In the last 7 days, Ms. Moore was never so weak that she had to limit her social activities, nor was her ever so week she couldn’t leave the house. She never needed help doing her usual activities because of fatigue and was never so tired she couldn’t take a short walk. She never needed to rest during the day.

In summary, Ms. Moore reports that she was:

• Never too weak to be limited in her social activities.

• Never too tired to leave the house.

• Never so weak she needed help doing her usual activities.

• Never too tired to take a short walk.

• Never so tired she needed to rest during the day.

Ms. Rivera’s Fatigue (T score = 37.5)

In the last 7 days, Ms. Rivera rarely felt fatigued or tired and never had to force herself to get up and do things because of being physically weak. She never had to limit her social activity because of feeling tired. She was never too tired to do household chores.

In summary, Ms. Rivera reports being:

• Never so weak that she had to force herself to get up and do things.

• Never so tired that it limited her social activity.

• Never so tired that it interfered with her household chores.

• Rarely fatigued.

• Rarely tired.

Mr. Perry’s Fatigue (T score = 42.5)

In the last 7 days, Mr. Perry was never so weak that he needed help doing his usual activities. He rarely had trouble starting or finishing things because he was too tired, and rarely needed to sleep during the day. He rarely felt like he had no energy.

In summary, Mr. Perry reports being:

• Never so weak he needed help doing usual activities.

• Rarely so tired he had trouble starting things.

• Rarely too tired he had trouble finishing things.

• Rarely so fatigued he needed to sleep during the day.

• Rarely so fatigued he felt he had no energy.

Ms. Miller’s Fatigue (T score = 47.5)

In the last 7 days, Ms. Miller rarely felt weak all over and rarely was so tired she couldn’t take a short walk. However, she sometimes felt tired, which got in the way of her doing her household chores. Feeling too tired to do the things she wanted to do was sometimes frustrating for her.

In summary, Ms. Miller reports being:

• Rarely weak all over.

• Rarely too tired to take a short walk.

• Sometimes tired.

• Sometimes too tired to do household chores.

• Sometimes frustrated by being too tired to do the things she wanted to do.

Ms. Lewis’ Fatigue (T score = 52.5)

In the last 7 days, Ms. Lewis was rarely so tired she couldn’t eat, but sometimes, her physical weakness caused her to have to force herself to get up and do things. Sometimes she was too tired to leave the house and had to limit her social activity because she was tired. Because of her fatigue, she sometimes needed help doing her usual activities.

In summary, Ms. Lewis reports she was:

• Rarely too tired to eat.

• Sometimes so weak she had to force herself to get up and do things.

• Sometimes too tired to leave the house.

• Sometimes so fatigued she needed help doing usual activities.

• Sometimes had to limit her social activity because of being tired.

Ms. Ross’ Fatigue (T score = 57.5)

In the last 7 days, Ms. Ross often felt exhausted and often was so tired she needed to rest during the day. She was often too tired to take a short walk or do household chores. Sometimes she needed to sleep during the day.

In summary, Ms. Ross reports being:

• Sometimes so tired she needed to sleep during the day.

• Often too tired to take a short walk.

• Often too tired to do household chores.

• Often so tired that she needed to rest during the day.

• Often exhausted.

Ms. King’s Fatigue (T score = 62.5)

In the last 7 days, Ms. King was sometimes too tired to eat. Because of her weakness, she often needed help doing her usual activities. She often felt weak all over, had no energy, and was always frustrated by being too tired to do the things she wanted to do.

In summary, Ms. King reports:

• Sometimes being too tired to eat.

• Often needing help doing usual activities because of weakness.

• Often being weak all over.

• Often having no energy.

• Always being frustrated because she was too tired to do the things she wanted to do.

Ms. Howard’s Fatigue (T score = 67.5)

In the last 7 days, Ms. Howard often had to limit his social activity because of physical weakness and was always exhausted. She always needed to sleep during the day. Ms. Howard was always too tired to take a short walk or do household chores.

In summary, Ms. Howard reports:

• Often being too physically weak for social activity.

• Always feeling exhausted.

• Always needing to sleep during the day.

• Always being too tired to take a short walk.

• Always being too tired to do household chores.

Mr. Nguyen’s Fatigue (T score = 72.5)

In the last 7 days, Mr. Nguyen always felt tired and without energy. He always needed help doing his usual activities and needed to limit his social activity because of fatigue. He was always frustrated by being too tired to do the things he wanted to do.

In summary, Mr. Nguyen reports:

• Always being tired.

• Always being without energy.

• Always needing help doing usual activities.

• Always needing to limit social activity because of being tired.

• Always feeling frustrated by being too tired to do the things he wanted to do.

Sleep

Mr. Flores’ Sleep (T score = 37.5)

In the last 7 days, Mr. Flores has rarely been sleepy during the daytime. He never has trouble staying awake during the day, and his daytime activities are never disturbed by poor sleep. Often, Mr. Flores feels like he has enough energy. He hasn’t had to force himself to get up in the morning.

In summary, Mr. Flores reported:

• Never having to force himself to get up in the morning

• Never having trouble staying awake during the day

• Never having his daytime activities disturbed by poor sleep

• Rarely being sleepy during the daytime

• Often feeling like he has enough energy

Mr. Turner’s Sleep (T score = 42.5)

In the last 7 days, Mr. Turner has rarely felt tired. Often, he has felt alert when he woke up and ready to start the day. In the last 7 days, he has never had trouble sleeping because of bad dreams and never had a hard time controlling his emotions because of poor sleep.

In summary, Mr. Turner reported:

• Never having trouble sleeping because of bad dreams

• Never having a hard time controlling his emotions because of poor sleep

• Rarely feeling tired

• Often feeling alert when he woke up

• Often waking up and feeling ready to start the day

Mr. Gómez’s Sleep (T score = 47.5)

During the last 7 days, Mr. Gómez has never had a hard time getting things done because of being sleepy. Rarely, he has had difficulty falling to sleep. Also rarely, he has had trouble stopping his thoughts at bedtime. Mr. Gómez says he has sometimes felt sleepy during the daytime, but often felt like he had enough energy.

In summary, Mr. Gómez reported:

• Never having a hard time getting things done because of being sleepy

• Rarely having difficulty falling asleep

• Rarely having trouble stopping his thoughts at bedtime

• Sometimes feeling sleepy during the daytime

• Often feeling he had enough energy

Mr. Harris’ Sleep (T score = 52.5)

During the last 7 days, Mr. Harris has rarely felt irritable because of poor sleep and rarely had difficulty waking up. Sometimes, he has felt like he had enough energy. Sometimes he felt alert when he woke up. In the past 7 days, he has never avoided or canceled activities with friends because he was tired from having a bad night’s sleep.

In summary, Mr. Harris reported:

• Never avoiding or canceling activities with friends because he was tired from a bad night’s sleep

• Rarely feeling irritable because of poor sleep

• Rarely having difficulty waking up

• Sometimes feeling alert when he woke up

• Sometimes feeling like he had enough energy

Mr. Wilson’s Sleep (T score = 57.5)

During the last 7 days, Mr. Wilson sometimes had pain that woke him up. Sometimes he had to force himself to get up in the morning and sometimes he woke up and felt ready to start the day. Rarely, did he have a hard time getting things done because of being sleepy. He never had trouble sleeping because of bad dreams.

In summary, Mr. Wilson reported:

• Never having trouble sleeping because of bad dreams

• Sometimes waking up because of pain

• Rarely having a hard time getting things done because he was sleepy

• Sometimes having to force himself to get up in the morning

• Sometimes waking up feeling ready to start the day

Mr. Murphy’s Sleep (T score = 62.5)

During the last 7 days, Mr. Murphy sometimes felt physically tense during the middle of the night or early morning hours. He often felt sleepy when he woke up and often felt tired. He sometimes had trouble staying awake during the day. Sometimes he had trouble concentrating because of poor sleep.

In summary, Mr. Murphy reported:

• Sometimes feeling physically tense during the middle of the night or early morning hours

• Sometimes having a hard time concentrating because of poor sleep

• Sometimes having trouble staying awake during the day

• Often feeling sleepy when he woke up

• Often feeling tire

Mr. Cook’s Sleep (T score = 67.5)

During the last 7 days, Mr. Cook was sometimes awoken by pain and often had problems during the day because of poor sleep. Also, his poor sleep often made it hard for him to control his emotions. Mr. Cook never felt alert when he woke up and never felt ready to start the day.

In summary, Mr. Cook reported:

• Sometime waking up because of pain

• Often having a hard time controlling his emotions because of poor sleep

• Often having problems during the day because of poor sleep

• Never feeling alert when he woke up

• Never waking up feeling ready to start the day

Ms. Roger’s Sleep (T score = 72.5)

During the last 7 days, Ms. Rogers often felt physically tense during the middle of the night or early morning hours. She often avoided or canceled activities with friends because of being tired from having a bad night’s sleep. She always had a hard time getting things done because of being sleepy, always tried to sleep whenever she could, and always had to force herself to get up in the morning.

In summary, Ms. Rogers reported:

• Often feeling physically tense during the middle of the night or early morning hours

• Often avoiding or canceling activities with her friends because she was tired from having a bad night’s sleep

• Always having a hard time getting things done because she was sleepy

• Always having to force herself to get up in the morning

• Always trying to sleep whenever she could

Ms. Smith’s Sleep (T score = 77.5)

During the last 7 days, Ms. Smith often had trouble sleeping because of bad dreams. Pain often woke her up. Ms. Smith always felt tired, and always had trouble staying awake during the day. Her poor sleep always disturbed her daytime activities.

In summary, Ms. Smith reported:

• Often having trouble sleeping because of bad dreams

• Often being awoken by pain

• Always having trouble staying awake during the day

• Always having his daytime activities disturbed by poor sleep

• Always feeling tired

Ms. Baker’s Sleep (T score = 82.5)

During the last 7 days, Ms. Baker often felt physically tense during the middle of the night or early morning hours. She always had a hard time controlling her emotions because of poor sleep. Ms. Baker always had difficulty waking up, always was sleepy during the daytime, and never had enough energy.

In summary, Mr. Baker reported:

• Often feeling physically tense during the middle of the night or early morning hours

• Always having a hard time controlling her emotions because of poor sleep

• Always having difficulty waking up

• Always being sleepy during the daytime

• Never having enough energy

Lower body function

Mr. Perez’s Lower Body Function (T score = 57.5)

Currently, Mr. Perez reports having no difficulty walking around one floor of his home, stepping up and down curbs, or getting out of bed and into a chair. He also has no difficulty pushing open a heavy door or running errands and shopping.

In summary, Mr. Perez reports:

• No difficulty walking around one floor of his home.

• No difficulty stepping up and down curbs.

• No difficulty getting out of bed and into a chair.

• No difficulty pushing open a heavy door.

• No difficulty running errands and shopping.

Ms. Adams’ Lower Body Function (T score = 52.5)

Currently, Ms. Adams reports having a little difficulty walking on a slippery, outdoor surface. She has no difficulty walking briskly for 20 min without stopping to rest. She also has no difficulty walking in a dark room without falling, going up and down a flight of stairs inside using a handrail, and no difficulty getting up off the floor from lying on her back without help.

In summary, Ms. Adams reports:

• A little difficulty walking on a slippery, outdoor surface.

• No difficulty walking briskly for 20 min without stopping to rest.

• No difficulty walking in a dark room without falling

• No difficulty going up and down a flight of stairs inside using a handrail

• No difficulty getting up off the floor from lying on her back without help.

Ms. Martinez’s Lower Body Function (T score = 47.5)

Currently, Ms. Martinez reports having a little difficulty climbing stairs step over step without using a handrail. She also has a little difficulty standing up from a low, soft couch. However, she reports having no difficulty getting in and out of a car, getting on and off the toilet, or walking for at least 15 min.

In summary, Ms. Martinez reports:

• A little difficulty climbing stairs step over step without using a handrail.

• A little difficulty standing up from a low, soft couch.

• No difficulty getting in and out of a car.

• No difficulty getting on and off the toilet.

• No difficulty walking for at least 15 min.

Ms. Myers’ Lower Body Function (T score = 42.5)

Currently, Ms. Myers reports having a little difficulty walking briskly for 20-min without stopping to rest. She has a little difficulty walking on uneven surfaces like grass, dirt roads and sidewalks. She also has a little difficulty walking in a dark room without falling. Finally, she reports having a little difficulty standing up from an armless straight chair (e.g., dining room chair) or pushing open a heavy door.

In summary, Ms. Myers reports:

• A little difficulty walking briskly for 20-min without stopping to rest.

• A little difficulty walking on uneven surfaces like grass, dirt roads, and sidewalks.

• A little difficulty walking in a dark room without falling.

• A little difficulty standing up from an armless straight chair (e.g., dining room chair)

• A little difficulty pushing open a heavy door

Ms. Hall’ Lower Body Function (T score = 37.5)

Currently, Ms. Hall reports some difficulty standing up from a low, soft couch. While indoors, she has a little difficulty going up and down a flight of stairs, even when using a handrail. If she doesn’t have help, Ms. Hall has much difficulty getting up from the floor if she’s lying on her back. She also reports a little difficulty going for a walk of at least 15 min. She reports no difficulty getting on and off the toilet.

In summary, Ms. Hall reports:

• When she doesn’t have help, she has much difficulty getting up from the floor from lying on her back.

• Some difficulty standing up from a low, soft couch.

• A little difficulty going up and down a flight of stairs inside using a handrail.

• A little difficulty going for a walk of at least 15 min

• No difficulty getting on and off the toilet.

Ms. Watson’s Lower Body Function (T score = 32.5)

Currently Ms. Watson reports a lot of difficulty walking in dark rooms without falling. She has some difficulty getting in and out of cars and running errands and shopping. Ms. Watson has a little difficulty getting on and off the toilet, and a little difficulty moving from sitting at the side of the bed to lying on her back on the bed.

In summary, Ms. Watson reports:

• A lot of difficulty walking in dark rooms without falling.

• Some difficulty getting in and out of cars.

• Some difficulty running errands and shopping.

• A little difficulty getting on and off the toilet

• A little difficulty moving from sitting at the side of the bed to lying down on her back.

Mr. Campbell’s Lower Body Function (T score = 27.5)

Currently Mr. Campbell reports having a lot of difficulty standing up from a low, soft couch, and a lot of difficulty walking on uneven surfaces like grass, dirt roads or sidewalks. He reports much difficulty pushing open a heavy door. He also has a lot of difficulty going up and down a flight of stairs while inside and using a handrail. Even when Mr. Campbell stays on one floor of his home, he has some difficulty walking around.

In summary, Mr. Campbell reports:

• A lot of difficulty standing up from a low, soft couch

• A lot of difficulty walking on uneven surfaces such as grass, dirt roads or sidewalks.

• Much difficulty pushing open a heavy door

• A lot of difficulty going up and down a flight of stairs while inside and using a handrail

• Some difficulty walking around one floor of his home.

Ms. Gray’s Lower Body Function (T score = 22.5)

Currently Ms. Gray reports that she has much difficulty getting in and out of a car, getting on and off the toilet, and getting out of bed and into a chair. When she has to sit down or stand up from a chair that has arms, she has a lot of difficulty. She also has a lot of difficulty moving from sitting at the side of the bed to lying on her back on the bed

In summary, Ms. Gray reports that she has:

• Much difficulty getting in and out of a car.

• Much difficulty getting on and off the toilet.

• Much difficulty getting out of bed and into a chair.

• A lot of difficulty sitting down on and standing up from a chair that has arms.

• A lot of difficulty moving from sitting at the side of the bed to lying on her back on the bed.

Ms. Cruz’s Lower Body Function (T score = 17.5)

Currently Ms. Cruz reports that she can’t walk briskly for 20-min without needing to stop for a rest. She also reports that she can’t walk on uneven surfaces and is unable to step up and down curbs. She is unable to run errands and shop and unable to get off the floor from lying on her back without help.

In summary, Ms. Cruz reports that she:

• Can’t walk briskly for 20-min without needing to stop for a rest.

• Can’t walk on uneven surfaces.

• Is unable to step up and down curbs.

• Is unable to run errands and shop.

• Is unable to get off the floor if she’s lying on her back unless she has help.

Upper body function

Ms. Phillips’ Upper Body Function (T score = 48.5)

Currently, Ms. Phillips has no difficulty shampooing her hair or brushing her teeth. She reports being able to write with a pen or pencil without difficulty. She is able to make phone calls using a touch tone key-pad without difficulty. She has no difficultly using a spoon when eating.

In summary, Ms. Phillips reports:

• No difficulty shampooing her hair

• No difficulty brushing her teeth

• No difficulty writing with a pen or pencil

• No difficulty making a phone call using a touch tone key-pad

• No difficulty using a spoon to eat a meal

Ms. Brooks’ Upper Body Function (T score = 43.5)

Currently, Ms. Brooks has a little difficulty cutting her toenails. She also reports a little difficulty removing wrappings from small objects. Likewise medications and vitamin containers (such as those with childproof lids and those that come in small bottles) are a little difficult for her to open. She has no difficulty opening jars that have already been opened once. She can wash and dry her body without any difficulty.

In summary, Ms. Brooks reports:

• A little difficulty cutting her toe nails

• A little difficulty removing wrappings from small objects

• A little difficulty opening medications or vitamin containers (e.g., childproof containers, small bottles)

• No difficulty opening previously opened jars

• No difficulty washing and drying her body

Mr. Cooper’s Upper Body Function (T score = 38.5)

Currently, Mr. Cooper has a little difficulty trimming his fingernails. He also has a little difficulty putting on a pullover shirt and bending down to pick up clothing from off the floor. He says he has no difficulty brushing his teeth and no difficulty writing with a pen or pencil.

In summary, Mr. Cooper reports:

• A little difficulty trimming his fingernails

• A little difficulty putting on a pullover shirt

• A little difficulty bending down and picking up clothing from the floor

• No difficulty writing with a pen or pencil

• No difficulty brushing his teeth

Mr. Wood’s Upper Body Function (T score = 33.5)

Currently, Mr. Wood has no difficulty using a spoon to eat a meal. Likewise, he has no difficulty making a phone call using a touch tone key-pad. He says he has a little difficulty opening and closing a zipper and some difficulty turning a key in a lock. He is able to button a shirt, but he does it with much difficulty.

In summary, Mr. Wood reports:

• No difficulty making a phone call or using a touch tone key-pad

• No difficulty using a spoon to eat a meal

• A little difficulty opening and closing a zipper

• Some difficulty turning a key in a lock

• Much difficulty buttoning a shirt

Mr. Young’s Upper Body Function (T score = 28.5)

Currently, Mr. Young has a little difficulty brushing his teeth and some difficulty washing and drying his body. He says he puts on his trousers with much difficulty and also has much difficulty opening medications or vitamin containers (e.g., childproof containers, small bottles). Mr. Young is unable to hold a plate that’s full of food.

In summary, Mr. Young reports:

• A little difficulty brushing his teeth

• A little difficulty putting on trousers

• Some difficulty washing and drying his body

• Much difficulty opening medications or vitamin containers (e.g., childproof containers, small bottles)

• Much difficulty pulling on his trousers

Ms. Fisher’s Upper Body Function (T score = 23.5)

Currently, Ms. Fisher has some difficulty using a spoon to eat a meal. She also has some difficulty making phone calls using a touch tone keypad. Ms. Fisher says she has much difficulty picking up coins from a table top. She is unable to take off a pullover shirt and also unable to remove wrappings from small objects.

In summary, Ms. Fisher reports:

• Some difficulty using a spoon to eat a meal

• Some difficulty making phone calls using a touch tone keypad

• Much difficulty picking up coins from a table top

• She is unable to remove wrappings from small objects

• She is unable to take off a pullover shirt

Mr. Bennett’s Upper Body Function (T score = 18.5)

Currently, Mr. Bennett has much difficulty writing with a pen or pencil. He reports being unable to do a number of things. He cannot trim his fingernails or shampoo his hair, nor can he button his shirt. He also is unable to open a jar that has been opened previously.

In summary, Mr. Bennett reports:

• Much difficulty writing with a pen or pencil

• Being unable to trim his fingernails

• Being unable to button his shirt

• Being unable to shampoo his hair

• Being unable to open a previously opened jar

Ms. Price’s Upper Body Function (T score = 13.5)

Currently, Ms. Price is unable to do many daily tasks that require upper body function. For example, she is unable to put on a pull over shirt and unable to cut her toe nails. In addition, she is not able to wash and dry her body or open and close a zipper. She is also unable to turn a key in a lock.

In summary, Ms. Price reports:

• Being unable to put on a pullover shirt

• Being unable to open and close a zipper

• Being unable to turn a key in a lock

• Being unable to wash and dry her body

• Being unable to cut her