Wednesday, December 31, 2008
Happy New Year!
Hope you're all ringing in the new year in a way that you enjoy, and that the new year brings peace and happiness to all.
Tuesday, December 30, 2008
On Travel
I am a homebody.
I really hate traveling. I hate sitting in the car or on an airplane for hours, awaiting arrival at my travel destination. I hate sleeping in strange beds, living out of a suitcase, or being someone's guest. I hate using strange showers and not having the right soap, shampoo, or conditioner. I hate not having the staples of my diet easily available or accessible.
Part of this is because my family always lived in the same house, with the exception of my dad's one-year sabbaticals to California and England. They had bought the house several years before I was born, and Dad still lives in my childhood home. While the stability was good, I think it also made it more difficult for me to adjust to changes in my living environment.
But sometimes, you have to travel. And sometimes, the fun outweighs the misery of travel, and it's worth it. For example, our Grand Canyon vacation was fun, and worth living out of a suitcase for a week, as was our recent holiday family time. Similarly, a good professional meeting or conference can cancel out all the travel misery.
Other times, the travel is not worth it. I do what I can to get out of that travel. I was supposed to attend a meeting next month in a very luxurious location, but I chose not to attend that meeting. The travel required would have been especially miserable -- a long flight followed by three hours in a shuttle van, and the same thing in reverse to get home. While it was tempting to go to this resort vacation spot for a one-day meeting, and do some tourist activities while I was there, I quickly realized that if I were going to go to that place, I would want to enjoy it with my family, not my colleagues. Since the meeting wasn't vital, I just decided to skip out.
Incidentally, I may not like traveling, but I do enjoy having guests.
I really hate traveling. I hate sitting in the car or on an airplane for hours, awaiting arrival at my travel destination. I hate sleeping in strange beds, living out of a suitcase, or being someone's guest. I hate using strange showers and not having the right soap, shampoo, or conditioner. I hate not having the staples of my diet easily available or accessible.
Part of this is because my family always lived in the same house, with the exception of my dad's one-year sabbaticals to California and England. They had bought the house several years before I was born, and Dad still lives in my childhood home. While the stability was good, I think it also made it more difficult for me to adjust to changes in my living environment.
But sometimes, you have to travel. And sometimes, the fun outweighs the misery of travel, and it's worth it. For example, our Grand Canyon vacation was fun, and worth living out of a suitcase for a week, as was our recent holiday family time. Similarly, a good professional meeting or conference can cancel out all the travel misery.
Other times, the travel is not worth it. I do what I can to get out of that travel. I was supposed to attend a meeting next month in a very luxurious location, but I chose not to attend that meeting. The travel required would have been especially miserable -- a long flight followed by three hours in a shuttle van, and the same thing in reverse to get home. While it was tempting to go to this resort vacation spot for a one-day meeting, and do some tourist activities while I was there, I quickly realized that if I were going to go to that place, I would want to enjoy it with my family, not my colleagues. Since the meeting wasn't vital, I just decided to skip out.
Incidentally, I may not like traveling, but I do enjoy having guests.
Monday, December 29, 2008
Like a Play in Which He's Reading My Lines
Me: Vinny, did you poo-poo?
Vinny: No. [Pause] I think he did.
Me: ...
(If he says that last line, we know he's pooped.)
Me: Would you like to read "Born"?*
Vinny: Okay. [Pause] Let's do it!
[At the table, after he's finished eating]
Vinny: I finish. I get down?
Me: Okay, let me get a washcloth.
Vinny: Good eatin'!
Me: It's time for sleepy-sleep. Let's go upstairs and brush teeth!
Vinny: Waaah! [Pause] I sowwy, it disappointing!
* On the Day You Were Born, by Debra Frasier, one of his favorite books.
Vinny: No. [Pause] I think he did.
Me: ...
(If he says that last line, we know he's pooped.)
Me: Would you like to read "Born"?*
Vinny: Okay. [Pause] Let's do it!
[At the table, after he's finished eating]
Vinny: I finish. I get down?
Me: Okay, let me get a washcloth.
Vinny: Good eatin'!
Me: It's time for sleepy-sleep. Let's go upstairs and brush teeth!
Vinny: Waaah! [Pause] I sowwy, it disappointing!
* On the Day You Were Born, by Debra Frasier, one of his favorite books.
Sunday, December 28, 2008
Holiday Travel
Did y'all miss me?
We were gone for a week to see both sides of the family in Kentucky for the holidays. We started with my dad and bonus mom, where my sisters, brother-in-law, and nephew all converged.
Before my siblings et al. arrived, we took the opportunity to see some old friends from high school and college. We went over to their house and had a great time catching up and eating pizza. The highlight of the evening for Vinny was their gigantic inflatable jump house in their basement that he could jump all over.
Once the family arrived, a good time was had by all. There was much food, much talking, and much laughter. My sisters and I had a girls' night out, and went to a museum we used to frequent as children, followed by a sushi dinner and then hot chocolate and deep conversation.
On Christmas Day, we headed over to the in-laws, where we had dinner and opened presents with Granny and Granddad and the fearless Aunt Ginger and Uncle Johnny. We spent the rest of our time there with the grandparents and with Jeff's aunt and uncle, who came up from their home in southeastern Kentucky for a visit.
It was nice to see all the family. This was the first time I didn't completely chicken out and isolate myself on Christmas Day itself. I had some special therapy which helped me to overcome the painful holiday memories of the past.
We were gone for a week to see both sides of the family in Kentucky for the holidays. We started with my dad and bonus mom, where my sisters, brother-in-law, and nephew all converged.
Before my siblings et al. arrived, we took the opportunity to see some old friends from high school and college. We went over to their house and had a great time catching up and eating pizza. The highlight of the evening for Vinny was their gigantic inflatable jump house in their basement that he could jump all over.
Once the family arrived, a good time was had by all. There was much food, much talking, and much laughter. My sisters and I had a girls' night out, and went to a museum we used to frequent as children, followed by a sushi dinner and then hot chocolate and deep conversation.
On Christmas Day, we headed over to the in-laws, where we had dinner and opened presents with Granny and Granddad and the fearless Aunt Ginger and Uncle Johnny. We spent the rest of our time there with the grandparents and with Jeff's aunt and uncle, who came up from their home in southeastern Kentucky for a visit.
It was nice to see all the family. This was the first time I didn't completely chicken out and isolate myself on Christmas Day itself. I had some special therapy which helped me to overcome the painful holiday memories of the past.
Thursday, December 25, 2008
Merry Christmas!
Hope your holiday is fun and enjoyable!
Monday, December 22, 2008
Sunday, December 21, 2008
Motherhood and Identity
Dr. Isis has a really profound post up on motherhood and maintaining your identity.
How can you make it as a successful scientist and a successful mother? There are a lot of women (as Dr. Isis describes in her post) who completely separate the work and personal spheres, have no pictures of their children up in the office, and deny that there is any difference between their current state of job devotion and their pre-children state.
Personally, I couldn't do that if I tried. My son is such an important part of my life that I can't imagine shutting down that part of my brain during the day. I do tell people at work what he's been up to lately, but I don't think I'm one of those people who can talk about nothing else. I mostly just recount some amusing vignettes and leave it at that.
He's not the first thing I talk about to everyone I meet at work, but if you work with me for long enough, you'll figure it out. I have a mini-shrine of baby pictures at my desk. My computer's screensaver is pictures of Vinny. And to anyone who asks I will tell a funny story about him.
Whether I like it or not, Vinny has changed the type of person I am, and I couldn't go back to pre-baby even if I wanted to. Like any major life event, the entrance of this little person into my life has been life-changing. I have a much different perspective now -- I don't sweat the small stuff so much, I feel more confident, and I'm a better leader.
Furthermore, I don't want to go back to that state. This has been a transformation for the better. He is an inspiration and an encouragement to me at work as well as at home.
I'd like to be an inspiration to other women who want to be scientists and mothers. Can you do both jobs well? Sure, it's just that you have to determine your priorities and be flexible about things that don't directly help you meet your goals. For example, having a spotless house is nice, but if cleaning takes up so much time that you don't have enough time to spend with your child, then adjusting your expectations may be a good idea.
Anyhow, I thought that Dr. Isis' post was really good and I'm glad that she is also keeping her scientist and mother identities integrated.
How can you make it as a successful scientist and a successful mother? There are a lot of women (as Dr. Isis describes in her post) who completely separate the work and personal spheres, have no pictures of their children up in the office, and deny that there is any difference between their current state of job devotion and their pre-children state.
Personally, I couldn't do that if I tried. My son is such an important part of my life that I can't imagine shutting down that part of my brain during the day. I do tell people at work what he's been up to lately, but I don't think I'm one of those people who can talk about nothing else. I mostly just recount some amusing vignettes and leave it at that.
He's not the first thing I talk about to everyone I meet at work, but if you work with me for long enough, you'll figure it out. I have a mini-shrine of baby pictures at my desk. My computer's screensaver is pictures of Vinny. And to anyone who asks I will tell a funny story about him.
Whether I like it or not, Vinny has changed the type of person I am, and I couldn't go back to pre-baby even if I wanted to. Like any major life event, the entrance of this little person into my life has been life-changing. I have a much different perspective now -- I don't sweat the small stuff so much, I feel more confident, and I'm a better leader.
Furthermore, I don't want to go back to that state. This has been a transformation for the better. He is an inspiration and an encouragement to me at work as well as at home.
I'd like to be an inspiration to other women who want to be scientists and mothers. Can you do both jobs well? Sure, it's just that you have to determine your priorities and be flexible about things that don't directly help you meet your goals. For example, having a spotless house is nice, but if cleaning takes up so much time that you don't have enough time to spend with your child, then adjusting your expectations may be a good idea.
Anyhow, I thought that Dr. Isis' post was really good and I'm glad that she is also keeping her scientist and mother identities integrated.
Saturday, December 20, 2008
Inquiring Minds Need to Know: Elmo Potty Time Edition
- Why does Elmo talk about himself in the third person? Is this supposed to resemble the way toddlers talk? Because mine never talks about himself in the third person; he talks about everyone else in the first person (e.g., I sit Mama = Mama sit).
- Would you feel like a failure if the apex of your artistic career was to be the singer of the "Dirty Diaper Blues" or the dude who raps about the manufacture of toilet paper?
- In addition to being furry like his son, Elmo's father has hair on his head and a mustache of a yellowish-orange color. He also has eyelids. Will Elmo develop those features?
- Elmo's dad also wears shirts. Why does Elmo go around naked?
- Where is Elmo's mom?
- Elmo talks about wearing big-kid underpants with Grover. Yet neither of them appear to wear clothes. How does that work?
- Who the heck is Baby Bear and why does he have that annoying speech impediment?
- Why does the big finale song get so stuck in my head that I wake up singing it?
Friday, December 19, 2008
Grammar Lesson of the Day
I was reading my usual blogs when I came across Dr. Mom's post on grammatical tips for writing, and while I did mention my most prominent pet peeve of the misplaced only, I decided to write this post rather than make an entire post-length comment on her blog.
I have heard this commercial on the radio too many times (well, more than zero times is too many times) where they say "All countertops are not created equal." Whenever I hear that I want to dig my ears out with a spoon!
The proper negation of "All countertops are created equal" is "Not all countertops are created equal" -- okay, people?!?!
The reason is this: let's say you're going around examining countertops, and you come across one that is unequal (whatever that even means!). Then, you have found a counterexample to the hypothesis of "All countertops are created equal." There could still be some countertops that are equal, but there exists at least one countertop that is unequal. Therefore you have shown that "Not all countertops are created equal," but since there exist some countertops that are still equal, you cannot say that "All countertops are not created equal," since clearly, there are some countertops that are equal.
Thursday, December 18, 2008
Christmas Presents
I'm not big on ritual or tradition. That's why Jeff and I just gave each other our Christmas presents when the packages arrived in the mail, rather than waiting for that special date.
For him: a new desktop computer. The old one had died, and he needed a replacement. This one isn't the latest model, but the price was right.
For me: a small digital camera. It's small enough that it fits in my purse. It's slightly larger than a deck of cards. There have been times when I've wished I had a camera available, to take a picture of some beautiful scenery, for example. This camera is perfect for my needs.
For Vinny: a big-boy bed and sheet set. The mattress is from Big Lots. The fully enclosing, waterproof vinyl mattress pad is from Wal-Mart. And the musical instrument-playing sea creatures comforter, sheet set, and throw pillows came from Sears. He has seen all the components -- the package from Sears is still languishing in our furniture-free living room -- and knows all about the big boy bed. We'll leave the Christmastime surprising to his grandparents, at least for this year.
For him: a new desktop computer. The old one had died, and he needed a replacement. This one isn't the latest model, but the price was right.
For me: a small digital camera. It's small enough that it fits in my purse. It's slightly larger than a deck of cards. There have been times when I've wished I had a camera available, to take a picture of some beautiful scenery, for example. This camera is perfect for my needs.
For Vinny: a big-boy bed and sheet set. The mattress is from Big Lots. The fully enclosing, waterproof vinyl mattress pad is from Wal-Mart. And the musical instrument-playing sea creatures comforter, sheet set, and throw pillows came from Sears. He has seen all the components -- the package from Sears is still languishing in our furniture-free living room -- and knows all about the big boy bed. We'll leave the Christmastime surprising to his grandparents, at least for this year.
Tuesday, December 16, 2008
Stay-at-Home Dads in the Area
Jeff attends Quaker Meeting on Sundays. For a while he took Vinny along, but Vinny has been in a screaming stage lately so these days he usually leaves Vinny with me. This past Sunday, Jeff couldn't find his wallet, so I drove him to the Quaker Meeting House and Vinny and I went shopping for an hour or two while Jeff participated in the Meeting.
When we came back, Jeff was involved in something-or-other and not quite ready to leave. While Vinny and I were waiting there, I got the opportunity to meet a stay-at-home dad and working mom couple who have a daughter who's about two years older than Vinny.
Jeff met this man and his family the first time he went to Quaker Meeting, because the man watches the children in the children's room during the service. I think Jeff was really happy to meet another stay-at-home dad, because this is the only other one we know. I know of another woman at my place of work who has a stay-at-home spouse, but I don't know her at all, having only seen her at women-oriented events.
It's nice to not be the only ones doing the reverse-traditional family arrangement, and I think we could be good friends with this other family. We should probably invite them over sometime.
When we came back, Jeff was involved in something-or-other and not quite ready to leave. While Vinny and I were waiting there, I got the opportunity to meet a stay-at-home dad and working mom couple who have a daughter who's about two years older than Vinny.
Jeff met this man and his family the first time he went to Quaker Meeting, because the man watches the children in the children's room during the service. I think Jeff was really happy to meet another stay-at-home dad, because this is the only other one we know. I know of another woman at my place of work who has a stay-at-home spouse, but I don't know her at all, having only seen her at women-oriented events.
It's nice to not be the only ones doing the reverse-traditional family arrangement, and I think we could be good friends with this other family. We should probably invite them over sometime.
Monday, December 15, 2008
Supercomputing Course: OpenMP Syntax and Directives, Part II
In order to be parallelizable by a parallel for construct, a loop must satisfy certain conditions. First, the number of iterations of the loop must be fixed, and knowable upon entry into the loop. Second, each iteration must be independent of the others. And finally, there must be no data dependence between iterations.
So, here are some things we can't parallelize with a parallel for construct:
In many scientific problems, we want to solve a system of linear equations. One of the ways to do this is known as Gaussian elimination. Suppose we want to parallelize the following Gaussian elimination code:
/* Gaussian Elimination (no pivoting): x = A\b */
for (int i = 0; i < N-1; i++) {
for (int j = i; j < N; j++) {
double ratio = A[j][i]/A[i][i];
for (int k = i; k < N; k++) {
A[j][k] -= (ratio*A[i][k]);
b[j] -= (ratio*b[i]);
}
}
}
We have three possible candidate loops to parallelize. Which one should we select?
First, we need to really understand how Gaussian Elimination works. For this, I refer to the below picture. (Click to embiggen.)
You can see the progression of Gaussian elimination in the four frames of this picture. The i loop is represented by the yellow row and column. Basically, we take the entries in the yellow row and column and use them to update the green submatrix before going on to row/column i+1. This tells us all we need to know about the parallelizeability of the i loop. The values of the entries in the (i+1)st yellow area depend on what operations were performed on them at previous values of i. Therefore we can't use OpenMP to parallelize this loop because of data dependence.
What about the j loop? The number of iterations in that loop varies with i, but we do know the number of iterations every time we are about to enter the loop. Do any of the later iterations depend on earlier ones? No, they do not. Can the j iterations be computed in any order? Yes, they can. So it looks like the j loop is parallelizable.
What about the k loop? Like the j loop, its number of iterations varies but is calculable for every i. None of the later iterations depend on earlier ones, and they can all be computed in any order. Therefore the k loop is also parallelizable.
So, we can't parallelize the outermost loop, but we can parallelize the inner two. Which one should we select? It's best to select the outer loop, because then we'll have more uninterrupted parallelism.
So, our Gaussian elimination code now looks like this (changes in red):
/* Gaussian Elimination (no pivoting): x = A\b */
for (int i = 0; i < N-1; i++) {
#pragma omp parallel for
for (int j = i; j < N; j++) {
double ratio = A[j][i]/A[i][i];
for (int k = i; k < N; k++) {
A[j][k] -= (ratio*A[i][k]);
b[j] -= (ratio*b[i]);
}
}
}
(Observe that we can combine the parallel and for constructs.)
Isn't that just amazing? We can parallelize this piece of code for the modest cost of one additional line! And our algorithm is still easy to read: It's not obscured by calls to OpenMP functions. Compare this to our old friend MPI. I never cease to be amazed by the compactness of OpenMP.
Next time, we'll talk about some other OpenMP directives, including sections, reduction, and some synchronization directives.
So, here are some things we can't parallelize with a parallel for construct:
- Conditional loops (e.g., many while loops; it's best to recast your parallelizable while loop as a for loop if you want to use the parallel for construct)
- Iterators over a loop (e.g., C++ STL iterators over a std::list)
- Loops in which any of the iterations are dependent upon any of the previous iterations
- Loops in which there is data dependence between iterations.
In many scientific problems, we want to solve a system of linear equations. One of the ways to do this is known as Gaussian elimination. Suppose we want to parallelize the following Gaussian elimination code:
/* Gaussian Elimination (no pivoting): x = A\b */
for (int i = 0; i < N-1; i++) {
for (int j = i; j < N; j++) {
double ratio = A[j][i]/A[i][i];
for (int k = i; k < N; k++) {
A[j][k] -= (ratio*A[i][k]);
b[j] -= (ratio*b[i]);
}
}
}
We have three possible candidate loops to parallelize. Which one should we select?
First, we need to really understand how Gaussian Elimination works. For this, I refer to the below picture. (Click to embiggen.)
You can see the progression of Gaussian elimination in the four frames of this picture. The i loop is represented by the yellow row and column. Basically, we take the entries in the yellow row and column and use them to update the green submatrix before going on to row/column i+1. This tells us all we need to know about the parallelizeability of the i loop. The values of the entries in the (i+1)st yellow area depend on what operations were performed on them at previous values of i. Therefore we can't use OpenMP to parallelize this loop because of data dependence.
What about the j loop? The number of iterations in that loop varies with i, but we do know the number of iterations every time we are about to enter the loop. Do any of the later iterations depend on earlier ones? No, they do not. Can the j iterations be computed in any order? Yes, they can. So it looks like the j loop is parallelizable.
What about the k loop? Like the j loop, its number of iterations varies but is calculable for every i. None of the later iterations depend on earlier ones, and they can all be computed in any order. Therefore the k loop is also parallelizable.
So, we can't parallelize the outermost loop, but we can parallelize the inner two. Which one should we select? It's best to select the outer loop, because then we'll have more uninterrupted parallelism.
So, our Gaussian elimination code now looks like this (changes in red):
/* Gaussian Elimination (no pivoting): x = A\b */
for (int i = 0; i < N-1; i++) {
#pragma omp parallel for
for (int j = i; j < N; j++) {
double ratio = A[j][i]/A[i][i];
for (int k = i; k < N; k++) {
A[j][k] -= (ratio*A[i][k]);
b[j] -= (ratio*b[i]);
}
}
}
(Observe that we can combine the parallel and for constructs.)
Isn't that just amazing? We can parallelize this piece of code for the modest cost of one additional line! And our algorithm is still easy to read: It's not obscured by calls to OpenMP functions. Compare this to our old friend MPI. I never cease to be amazed by the compactness of OpenMP.
Next time, we'll talk about some other OpenMP directives, including sections, reduction, and some synchronization directives.
Sunday, December 14, 2008
Supercomputing Course: OpenMP Syntax and Directives, Part I
The OpenMP standard covers the C, C++, and Fortran programming languages. There is one syntax for C and C++, and another for Fortran. In this tutorial, I will focus on the C/C++ syntax.*
The basic format of an OpenMP directive is as follows:
#pragma omp directive-name [clause] newline
(Incidentally, the word pragma is Greek for "thing.") If you have a compiler that doesn't understand OpenMP, it will simply ignore the pragma lines, making it possible to use the same code file for both OpenMP-parallelized and serial code.**
A few more points about the syntax:
#pragma omp parallel private(list) shared(list)
{
/* put parallel code here */
}
The parallel directive is used to create a block of code that is executed by multiple threads. The code in brown are optional arguments to the directive -- private and shared variables, which we'll talk about later.
Here is a sample code -- the OpenMP equivalent of "Hello World":
#include <stdio.h>
#include <omp.h>
int main (int argc, char *argv[]) {
int tid;
printf("Hello world from threads:\n");
#pragma omp parallel private(tid)
{
tid = omp_get_thread_num();
printf("(%d)\n", tid);
}
printf("I am sequential now\n");
return 0;
}
Suppose we run this program using four threads. What will be the output of this example?
may be the answer you're counting on, but it may not be what you receive. The output
is just as valid and just as likely. The point being, the operating system schedules the threads, and you should not count on them executing in any particular order.
The most common directive other than the parallel section directive (which you must use in every program to contain the parallel section of code) is the parallel for directive. The syntax is as follows:
#pragma omp for schedule(type [,chunk]) \
private(list) shared(list) nowait
{
/* for loop */
}
where type = {static, dynamic, guided, runtime}. If nowait is specified, then the threads won't wait around for all threads to finish up the for loop before moving on to the next thing.
The default scheduling is implementation dependent. The scheduling choices are somewhat intuitive in meaning. Static scheduling simply means that the ID of the thread that is assigned a given iteration is a function of the iteration number and the number of threads, and is determined at the beginning of the loop. This is a simple way to assign iterations, but it could be a bad idea if different iterations have different amounts of work to them. For example, if every fourth iteration has extra work, and we have four threads, then a certain thread will be assigned that work-laden iteration every time, and we will have a load imbalance.
In that case, the dynamic scheduling algorithm may be a better choice. The threads are assigned iterations of the loop over the course of the execution of the loop, in a round robin fashion. So, a thread that is idle gets the next piece of work to do, meaning that threads are assigned work based on their ability to complete the work. In this way, load balance can occur.
You are probably wondering how to use the parallel for construct, but this entry has gotten very long so I will stop here for now. Next time, we will discuss the parallel for further, and I will provide an example.
* This is because as a computer scientist, I think that C and C++ are pretty nice languages, and Fortran, which in a just world would be nothing more than a fascinating relic of the past, is an undead zombie language that refuses to die.
** This is true for OpenMP directives, but if you use any of the runtime library routines, you will need to do a little more to make things work. We will talk about this when we talk about runtime library routines.
The basic format of an OpenMP directive is as follows:
#pragma omp directive-name [clause] newline
(Incidentally, the word pragma is Greek for "thing.") If you have a compiler that doesn't understand OpenMP, it will simply ignore the pragma lines, making it possible to use the same code file for both OpenMP-parallelized and serial code.**
A few more points about the syntax:
- Note the newline at the end of the directive line. We will see some pragma that require braces around the code to be parallelized. You must put the beginning curly brace on the line after the directive, not on the same line as the directive like many people do in the case of for loops.
- The pragma are case sensitive, and follow all the standard C/C++ pragma rules.
- If your directive is really long, you can continue it on the next line by putting a backslash as the final character in the previous line.
#pragma omp parallel private(list) shared(list)
{
/* put parallel code here */
}
The parallel directive is used to create a block of code that is executed by multiple threads. The code in brown are optional arguments to the directive -- private and shared variables, which we'll talk about later.
Here is a sample code -- the OpenMP equivalent of "Hello World":
#include <stdio.h>
#include <omp.h>
int main (int argc, char *argv[]) {
int tid;
printf("Hello world from threads:\n");
#pragma omp parallel private(tid)
{
tid = omp_get_thread_num();
printf("(%d)\n", tid);
}
printf("I am sequential now\n");
return 0;
}
Suppose we run this program using four threads. What will be the output of this example?
Hello world from threads:
(0)
(1)
(2)
(3)
I am sequential now
may be the answer you're counting on, but it may not be what you receive. The output
Hello world from threads:
(3)
(1)
(0)
(2)
I am sequential now
is just as valid and just as likely. The point being, the operating system schedules the threads, and you should not count on them executing in any particular order.
The most common directive other than the parallel section directive (which you must use in every program to contain the parallel section of code) is the parallel for directive. The syntax is as follows:
#pragma omp for schedule(type [,chunk]) \
private(list) shared(list) nowait
{
/* for loop */
}
where type = {static, dynamic, guided, runtime}. If nowait is specified, then the threads won't wait around for all threads to finish up the for loop before moving on to the next thing.
The default scheduling is implementation dependent. The scheduling choices are somewhat intuitive in meaning. Static scheduling simply means that the ID of the thread that is assigned a given iteration is a function of the iteration number and the number of threads, and is determined at the beginning of the loop. This is a simple way to assign iterations, but it could be a bad idea if different iterations have different amounts of work to them. For example, if every fourth iteration has extra work, and we have four threads, then a certain thread will be assigned that work-laden iteration every time, and we will have a load imbalance.
In that case, the dynamic scheduling algorithm may be a better choice. The threads are assigned iterations of the loop over the course of the execution of the loop, in a round robin fashion. So, a thread that is idle gets the next piece of work to do, meaning that threads are assigned work based on their ability to complete the work. In this way, load balance can occur.
You are probably wondering how to use the parallel for construct, but this entry has gotten very long so I will stop here for now. Next time, we will discuss the parallel for further, and I will provide an example.
* This is because as a computer scientist, I think that C and C++ are pretty nice languages, and Fortran, which in a just world would be nothing more than a fascinating relic of the past, is an undead zombie language that refuses to die.
** This is true for OpenMP directives, but if you use any of the runtime library routines, you will need to do a little more to make things work. We will talk about this when we talk about runtime library routines.
Saturday, December 13, 2008
Propaganda
We have begun a propaganda campaign in this household. The campaign is pro-potty (or "polly" -- as it's referred to around here).
We have the DVD "Elmo Potty Time," and it is quite popular with the demographic to which this campaign is directed. Unfortunately for us, Elmo's high-pitched voice gives us a headache, and his laugh sounds evil. But perhaps that is what makes Elmo so successful -- he is evil, and uses his evil superpowers on your children.
At this point, we don't really care. Whatever gets our 32-pound karate-kicking expert to use the potty so we no longer need to dodge his whirling feet at the changing table, we're all for it.
We have the DVD "Elmo Potty Time," and it is quite popular with the demographic to which this campaign is directed. Unfortunately for us, Elmo's high-pitched voice gives us a headache, and his laugh sounds evil. But perhaps that is what makes Elmo so successful -- he is evil, and uses his evil superpowers on your children.
At this point, we don't really care. Whatever gets our 32-pound karate-kicking expert to use the potty so we no longer need to dodge his whirling feet at the changing table, we're all for it.
Thursday, December 11, 2008
By Thy Money Gently Flowing, Illinois, Illinois*
As I'm sure you've all heard by now, the governor of Illinois was arrested on Tuesday morning for corruption, including attempting to use the filling of Barack Obama's empty senate seat for personal gain.
Wow.
Dang.
I must admit it: I voted for Blagojevich in 2002, but by the time he was up for re-election, I had moved to Tennessee.
That makes two completely corrupt, lawbreaking frauds we elected in a row (George Ryan, and now Blagojevich). What other state can say that?!?!
It's funny, because when I moved to Illinois from Kentucky, shortly after Operation Boptrot, I was relieved to be moving away from such a corrupt state. How wrong I was!
Wednesday, December 10, 2008
Adventures with Vinny
This past weekend, we had a lot of fun with Vinny. He loves the trains at our local children's museum, and on Saturday night, they had a special showing of the larger model trains in their outdoor train garden. The trains were decorated in a Christmas theme, and all the little buildings had lights on them. Vinny loved seeing the trains, and if it hadn't been so darn cold we would have stayed there just watching the trains the whole time.
Then the next day, we took him back to the museum during daytime hours, so he could check out the H-O scale model trains inside, plus all the other stuff he loves in the museum (such as the nests that light up when he pushes the coordinating buttons). It was also our intention to let him run around and wear himself out. But it was a lot of fun because he was just so thrilled to be there and to see all the interesting stuff.
On Monday, Jeff was feeling a little under the weather so I took it upon myself to cook dinner. Vinny helped me make some pancakes. He's really interested in cooking, and he especially loves the mixer. We didn't use the mixer but we did use a whisk, which he also finds fascinating. I let him help me by dumping the measuring cups of flour into the bowl, and holding the end of the whisk handle while I stirred the batter up. He was really eager to help -- "I help me, Mama?" he offered.
I let him stand on a step-stool while I cooked the pancakes on the griddle in the middle of our stovetop. I sufficiently impressed upon him the fact that it was hot, such that he didn't even try to touch it. But he did watch in fascination as the pancakes cooked. We made BIIIGGGGG PANCAKES (said with great breadth and in a low voice) and itty-bitty pancakes (said in a thin, high voice) and he ate probably 4 of the smaller pancakes while we were standing there making pancakes. Then at the table he ate another two.
When I come home from work every day he is always delighted to see me. He hugs me and regularly says "I miss you all day." Today he was working on a very intricate drawing -- "I draw stars!" he said -- which was a new development. Normally he just scribbles a bit and moves on to the next piece of paper, but according to Jeff he'd been working on this one all day. It is completely abstract -- nothing recognizable at all, no repetitions or patterns -- but really quite stunning. I will have to take a picture of his masterpiece.
Tuesday, December 09, 2008
Supercomputing Course: OpenMP
OpenMP is the industry standard shared memory programming model. First developed in 1997, the OpenMP standard is updated by an architecture review board (ARB).
There are several really great advantages to OpenMP. First, it allows you to take an incremental approach to parallelization. You can parallelize the the parts that will have the most impact before parallelizing the rest, optimizing your effort-to-improvement ratio. Second, the constructs you use to parallelize the code are simple to read, and most are one-liners, so your code doesn't grow that much. And finally, you can use the same code as parallel or serial code -- just compile it with or without OpenMP enabled, depending on what you want.
The OpenMP API is a combination of directives, runtime library routines, and environment variables. These components of the API fall into three categories: expression of parallelism, data sharing among threads, and synchronization.
The OpenMP parallelism model is a fork-and-join model. That is, when your program starts running, there is a single thread of control. When it reaches a parallel section, it forks into multiple threads, which rejoin into the single thread upon reaching the end of the parallel section.
The nuclear physicist in our previous installment could write a hybrid MPI/OpenMP program to solve a bigger problem. She would reproduce the basis functions once on each node, and then use OpenMP to parallelize the computations across the node. As we will see in the next installment of this course, there are many OpenMP directives that can be used to parallelize many common features of algorithms.
There are several really great advantages to OpenMP. First, it allows you to take an incremental approach to parallelization. You can parallelize the the parts that will have the most impact before parallelizing the rest, optimizing your effort-to-improvement ratio. Second, the constructs you use to parallelize the code are simple to read, and most are one-liners, so your code doesn't grow that much. And finally, you can use the same code as parallel or serial code -- just compile it with or without OpenMP enabled, depending on what you want.
The OpenMP API is a combination of directives, runtime library routines, and environment variables. These components of the API fall into three categories: expression of parallelism, data sharing among threads, and synchronization.
The OpenMP parallelism model is a fork-and-join model. That is, when your program starts running, there is a single thread of control. When it reaches a parallel section, it forks into multiple threads, which rejoin into the single thread upon reaching the end of the parallel section.
The nuclear physicist in our previous installment could write a hybrid MPI/OpenMP program to solve a bigger problem. She would reproduce the basis functions once on each node, and then use OpenMP to parallelize the computations across the node. As we will see in the next installment of this course, there are many OpenMP directives that can be used to parallelize many common features of algorithms.
Monday, December 08, 2008
Supercomputing Course: Scalability, Memory, and Multicore Architecture
Remember my supercomputing course? Good! Here's another installment.
We discussed scalability before. To review, there are two primary measures of scalability of your program: strong scaling, in which we throw different numbers of processors at a constant-sized problem, and hope that the wall time to completion is inversely proportional to the number of processors; and weak scaling, in which we throw a constant amount of work per process at varying numbers of processors, and hope that the wall time to completion remains constant.
Scalability is a very handy concept, because it alerts us to bottlenecks in our algorithms. We may not know exactly what's wrong, but if we see that the numbers are not what we would like them to be, we can go in and check it out.
Sometimes, however, we get some false readings from our scaling studies. For example, I once had superlinear speedup, meaning that in my strong scaling study, I found that it took less than half the wall time on 2N processors that it did on N processors. This was because of caching effects -- a (2N)th of the problem fit fully into cache memory, while an Nth did not. Since the machine never had to access the slower memory, the computation went even faster than it would have otherwise.
Speaking of memory, to solve some problems we have to duplicate a data set across every process. For example, a nuclear physicist might need to store her basis functions on every process, because the basis functions are used in the computation of potentials for every particle in the nucleus. This can be very redundant, and especially in the case of today's mixed distributed and multicore supercomputer architecture, it is also a waste of memory space.
On a multicore supercomputer, instead of each processor consisting of a single CPU plus memory, we have multiple CPUs that physically share one large chunk of memory. If we use MPI, then we treat each CPU as if it is a single CPU with memory that is inaccessible to all the other processes, even the ones that are on the same board sharing the same memory pool with it. So in this scenario, the nuclear physicist would have to reproduce the same data twice for a dual-core node, four times for a quad-core node, or eight times for an eight-core node. For example, on a quad-core machine with 2 GB memory per core, if our basis functions take up 1 GB, then the redundant basis function storage occupies 4 out of the 8 GB available. Wouldn't it be cool if we could just reproduce it once? We could use that extra 3 GB to solve a bigger problem.
We could use only one of the cores on the node, letting the rest remain unused, and use all the memory on the node for that one processor. Unfortunately, any computations that we perform will be done on only one of the cores, thereby slowing them by a factor of n for an n-core node.
What we need instead is a way to take advantage of all the memory while simultaneously using all the processors when we do computations. In other words, we want to use a distributed memory programming model between multicore nodes, and a shared memory programming model within the nodes. Is there a shared memory programming model that we could use for this purpose?
Of course there is; it's called OpenMP, and it's the topic of the next installment of this course!
We discussed scalability before. To review, there are two primary measures of scalability of your program: strong scaling, in which we throw different numbers of processors at a constant-sized problem, and hope that the wall time to completion is inversely proportional to the number of processors; and weak scaling, in which we throw a constant amount of work per process at varying numbers of processors, and hope that the wall time to completion remains constant.
Scalability is a very handy concept, because it alerts us to bottlenecks in our algorithms. We may not know exactly what's wrong, but if we see that the numbers are not what we would like them to be, we can go in and check it out.
Sometimes, however, we get some false readings from our scaling studies. For example, I once had superlinear speedup, meaning that in my strong scaling study, I found that it took less than half the wall time on 2N processors that it did on N processors. This was because of caching effects -- a (2N)th of the problem fit fully into cache memory, while an Nth did not. Since the machine never had to access the slower memory, the computation went even faster than it would have otherwise.
Speaking of memory, to solve some problems we have to duplicate a data set across every process. For example, a nuclear physicist might need to store her basis functions on every process, because the basis functions are used in the computation of potentials for every particle in the nucleus. This can be very redundant, and especially in the case of today's mixed distributed and multicore supercomputer architecture, it is also a waste of memory space.
On a multicore supercomputer, instead of each processor consisting of a single CPU plus memory, we have multiple CPUs that physically share one large chunk of memory. If we use MPI, then we treat each CPU as if it is a single CPU with memory that is inaccessible to all the other processes, even the ones that are on the same board sharing the same memory pool with it. So in this scenario, the nuclear physicist would have to reproduce the same data twice for a dual-core node, four times for a quad-core node, or eight times for an eight-core node. For example, on a quad-core machine with 2 GB memory per core, if our basis functions take up 1 GB, then the redundant basis function storage occupies 4 out of the 8 GB available. Wouldn't it be cool if we could just reproduce it once? We could use that extra 3 GB to solve a bigger problem.
We could use only one of the cores on the node, letting the rest remain unused, and use all the memory on the node for that one processor. Unfortunately, any computations that we perform will be done on only one of the cores, thereby slowing them by a factor of n for an n-core node.
What we need instead is a way to take advantage of all the memory while simultaneously using all the processors when we do computations. In other words, we want to use a distributed memory programming model between multicore nodes, and a shared memory programming model within the nodes. Is there a shared memory programming model that we could use for this purpose?
Of course there is; it's called OpenMP, and it's the topic of the next installment of this course!
Sunday, December 07, 2008
Apparently, He Understands Sequence, but Not Consequence
Typical scenes in our lives:
Vinny: [Screams loudly]
[Then, in a stage whisper:] Shhh! Be quiet!
****
[Vinny begins climbing up on a chair in order to reach the light switch.]
Vinny: Be careful!
Vinny: [Screams loudly]
[Then, in a stage whisper:] Shhh! Be quiet!
****
[Vinny begins climbing up on a chair in order to reach the light switch.]
Vinny: Be careful!
Saturday, December 06, 2008
Productive Week
Despite the fact that I took a sick day on Monday, and felt crappy for much of the remainder of the week, I accomplished a lot at work last week. I successfully compiled and ran a code I'd been trying to get working for three weeks, which felt really good. Also, I did some performance runs on my main project that I'd been meaning to do for a really long time and finally got around to doing. And finally, I started working on implementing some I/O for one of my other projects.
I don't know if I have it working right, but I was pleased that I got a good start on it. This project is particularly challenging because I don't really understand what the code is doing, and it's written in Fortran (not my first choice of programming languages, let me tell you...). What I have to do for them is to convert their checkpointing I/O from MPI-IO to another type of I/O. This is hard because this new type of I/O is completely foreign to me; in fact, I am kind of a guinea pig because I am the first person outside of the developers to try to use it. Luckily the developers live down the hall from me, so I can just talk to them when I run into trouble. The documentation isn't very clear yet (because, as I said, I am a beta tester) and I will have lots of feedback for them once I finally figure out what I'm doing.
But I can see that this new I/O system is extremely powerful and will make people's lives a lot easier. Something else that's bogging me down in converting from MPI-IO is deciphering from the I/O subroutines just what it is that the code developers want to be output and input. An analogy I used yesterday is this: I know that they want to write a capital letter of some sort, but instead of them just saying "I want to write out the letter A," I instead have only clues about how the letter is constructed: "A diagonal line beginning from the bottom and going up and to the right; another diagonal line beginning where that one ends and going down and to the left; and a straight line left-to-right connecting the two at the midline." From that, I follow the instructions and realize that they're writing a capital A. But it is hard to decipher.
Luckily I have a friend and colleague who works in this field and is proficient in Fortran. She's also supposed to be working with me on this project, although she works on different aspects than I do. So I have talked to her and had her help me decipher what they are trying to do. In one case, she was able to crack the code and figure out what the heck was going on, and determined that they were doing the output really inefficiently, kind of like saying "create a diagonal line from the bottom left corner to the midline, then create a diagonal line from the top and center down to the right, to the midline, then create a diagonal line up and to the right from where you ended the first stroke on the midline, then create a straight line left-to-right connecting the two diagonal lines at the midline, and then create a diagonal line from the right end of the midline horizontal line down to the lower right corner." Yes, that makes a capital A, but it's a kind of inefficient way of going about it.
So anyhow, I should be ready to start compiling this new code sometime next week. I'm pretty excited about it, although I know that there will be many errors to fix. But it feels good because I know this work has the potential to really transform their science capabilities.
I don't know if I have it working right, but I was pleased that I got a good start on it. This project is particularly challenging because I don't really understand what the code is doing, and it's written in Fortran (not my first choice of programming languages, let me tell you...). What I have to do for them is to convert their checkpointing I/O from MPI-IO to another type of I/O. This is hard because this new type of I/O is completely foreign to me; in fact, I am kind of a guinea pig because I am the first person outside of the developers to try to use it. Luckily the developers live down the hall from me, so I can just talk to them when I run into trouble. The documentation isn't very clear yet (because, as I said, I am a beta tester) and I will have lots of feedback for them once I finally figure out what I'm doing.
But I can see that this new I/O system is extremely powerful and will make people's lives a lot easier. Something else that's bogging me down in converting from MPI-IO is deciphering from the I/O subroutines just what it is that the code developers want to be output and input. An analogy I used yesterday is this: I know that they want to write a capital letter of some sort, but instead of them just saying "I want to write out the letter A," I instead have only clues about how the letter is constructed: "A diagonal line beginning from the bottom and going up and to the right; another diagonal line beginning where that one ends and going down and to the left; and a straight line left-to-right connecting the two at the midline." From that, I follow the instructions and realize that they're writing a capital A. But it is hard to decipher.
Luckily I have a friend and colleague who works in this field and is proficient in Fortran. She's also supposed to be working with me on this project, although she works on different aspects than I do. So I have talked to her and had her help me decipher what they are trying to do. In one case, she was able to crack the code and figure out what the heck was going on, and determined that they were doing the output really inefficiently, kind of like saying "create a diagonal line from the bottom left corner to the midline, then create a diagonal line from the top and center down to the right, to the midline, then create a diagonal line up and to the right from where you ended the first stroke on the midline, then create a straight line left-to-right connecting the two diagonal lines at the midline, and then create a diagonal line from the right end of the midline horizontal line down to the lower right corner." Yes, that makes a capital A, but it's a kind of inefficient way of going about it.
So anyhow, I should be ready to start compiling this new code sometime next week. I'm pretty excited about it, although I know that there will be many errors to fix. But it feels good because I know this work has the potential to really transform their science capabilities.
Friday, December 05, 2008
Scientiae Carnival Posted
For those of you who are fans of hot science, might I recommend the latest installment of the Scientiae Carnival? Dr. Isis has carefully and lovingly included many very fascinating posts detailing the hotness of many diverse scientific areas. Thanks, Dr. Isis, for such a great Scientiae Carnival!
Thursday, December 04, 2008
Typhoid Vinny
I love my son, truly. He is the sweetest child ever, and he's so thoughtful, generous, and caring. But I must admit, I could do without the generosity when it comes to sharing illness.
After you share so many glasses of water, bowls, and eating utensils; after you get touched by snotty fingers enough; and get coughed in your face enough, eventually you will succumb to the illness. And succumb I have!
I went to bed really early on Sunday night. But when I got up the next day, I felt really bad so I stayed home from work. Tuesday was a little better, so I went ahead and went to work, but by the time I got home I was pretty much dead to the world and went to bed at about 8 pm. Yesterday was a lot better, so I think I am on the home stretch in terms of getting well.
Tuesday, December 02, 2008
Inquiring Minds Need to Know: Oswald Edition
Important questions about the world of Oswald:
- How does Oswald the Octopus survive outside of water?
- Why does an octopus need a life preserver or flippers when going in the water?
- What holds up Oswald's sunglasses, seeing as he has no nose or ears?
- Is Daisy anorexic? (She's very thin, always exercising, and almost never eats -- for example, when she and Henry came over to Oswald's for hot cocoa, she had peppermint tea instead of cocoa.)
- Is it cannibalism when Daisy drinks peppermint tea or eats sunflower seeds?
- Also, does Daisy ever sleep? She was the only person to wake up when the town clock stopped.
- Why does Henry the Penguin hate water?
- What happened in Henry's life that made him develop OCD and narcissistic personality disorder?
- Did Oswald's insurance premiums go up after he popped so much popcorn that it filled up his apartment and spilled down the steps, and put so much bubble solution in the wading pool outside that the froth expanded across the yard and into the building?
- The bottom of Oswald's apartment window is at the same level as the top of Henry's window in the same building. Are the levels of the building staggered?
- Did the turtle who was seen ascending the stairs from the time Oswald went to Henry's for a sleepover to the time they went to sleep, and descending the stairs in the morning, get any sleep?
- Why does everything cost exactly one coin -- anything from a single marshmallow to a bucket of paint?
- Where does Oswald get his money?
- After the number of times that Madame Butterfly has lost her daughter Catrina and Oswald has rescued her, why hasn't Child Protective Services taken Catrina away?
- And what does single mother Madame Butterfly do with Catrina when she has to work?
- How does Johnny the Snowman not melt?
- Is it cannibalism that Johnny operates an ice cream shop?
- What does Oswald do to his plants to make them grow so quickly, e.g., his banana seeds that yielded a gigantic banana overnight; his tomato plants that produced ripe tomatoes in less than 12 hours?
- He has the same talent with animals. How did he get his pet goldfish to grow from inch-size to bathtub-size in under a week?
Monday, December 01, 2008
November Experiment
As you may have observed, I posted a lot last month. I wanted to see whether I was capable of posting once a day while still conforming to my rules of computer use.
It seems like I was successful. I confess that I cheated and used the advanced post scheduling feature for at least half the posts. I don't think I will keep up this pace, but maybe I'll post a little more often than before.
It seems like I was successful. I confess that I cheated and used the advanced post scheduling feature for at least half the posts. I don't think I will keep up this pace, but maybe I'll post a little more often than before.
Subscribe to:
Posts (Atom)