[Eisman] called Standard & Poor’s and asked what would happen to default rates if real estate prices fell. The man at S&P couldn’t say; its model for home prices had no ability to accept a negative number. “They were just assuming home prices would keep going up,” Eisman says.
—“The End”, Michael Lewis, Portfolio.com, 11 Nov 2008

Yes, those unbelievably savvy people at S&P, whose ratings department are/were supposed to be trusted with assessing the risk of investments/credit lines, simply didn’t have a “model” (that is, some black-box computer program of a mathematical formula some analyst thought would predict market movements) that could cope with the concept that house prices might, at some point in the future, not rise.

(Also, this is long. Condensed: I’ve been using Freebase to store my reading data, I wrote an Acre app to provide a custom view, and I discovered that my data model has some shortcomings.)

I’ve been playing with Acre some more, specifically on a long-term project of mine: to store data about the books I read in some system and then create views about my reading habits. Yes, compulsive list-making combined with programming/data geekery.

Anyway, I could have used a lot of other systems, such as Delicious Library or LibraryThing or Books, to store this information, but none of them seemed to have quite what I want (and most of them are proprietary). I could have written my own, and planned to, but kept tweaking with the data model and generally wasn’t sure how I wanted to deal with it.

I was going down the path of having objects for Books, instances of Books, Book reading events, and Authors. Then I encountered Freebase—I said that this has been a long-term project—and decided that I like the data model in its Publishing domain.

The relevant Freebase types for me are:
Book
Book Edition
Written Work
Author

Written Work is an included type for Book, so whenever you assert that something is a Book, the Freebase frontend asserts that it is also a Written Work. I considered Written Work to essentially be part of Book for my purposes.

I’ve been keeping track, in text files, the books that I read for quite some time. I’ve gradually increased the information I gather about each one, so that what was originally just date started, date finished, title, author, and whether or not I’d read the book before, has become this:

07. 29/03/2008; Matter; Iain M. Banks
593 pages; started 28/03/2008; 2008; Orbit, New York Feb 2008; ISBN-10: 0316005363; ISBN-13: 9780316005364; FBG: 9202a8c04000641f80000000091ceee6;

The last part, the most recent addition, stands for FreeBase Guid—that is, if you tack /guid/ onto the front of it, you have a Freebase identifier. One of many ways to use this is to just put the Freebase view address in front of it, like so: http://www.freebase.com/view/guid/9202a8c04000641f80000000091ceee6, which gives you the link to the Freebase page for the edition of Matter that I read in March this year.

The reason that edition is in there is because I added it. Freebase doesn’t have fantastic coverage for book data yet (thought that may change), so at the moment I manually enter the edition data, and occasionally the book data, for each book I read.

So, once all that is in the system, there should be some way for me to tag the books that I’ve read. The easiest way to do this would be to create a type, something like “Books Tadhg has read”, with the date properties. I don’t like that approach because Freebase types are a little like tags, and my addition of this type to a topic would show up for everyone, which strikes me as a little like pollution (because I assume that details on what I’ve been reading are not as fascinating to everyone as they are to me and, no doubt, the readers of this blog).

Instead, I created the types Book Edition Reader and Book Edition Reading Event. Before going into that, let me explain why I used Book Edition instaed of Book—it’s mainly because I want the easiest path to the data that’s relevant to me, which is the data that I capture myself as in those two lines above indicating when I read Matter, how long it was, where it was published, and so on. Because a Book Edition can refer to only one Book, it seemed an easier path to specify just the Book Edition, from which the Book can later be extracted, whereas if I specified Book, I would also have to specify which Book Edition of that Book I meant. Book Edition, then, was the approach.

My approach is more convoluted than adding a type to every Book Edition, but still fairly simple. The Book Edition Reader type is attached to a person, and takes Book Edition Reading Event in a property that expects a list of things. Book Edition Reading Event has the reader (me, since nobody else has used this so far, although it’s there for anyone who wants it), the start date, the end date, and the Book Edition read.

Simple. Once it’s all in there, I can get the list of Book Editions I’ve read with a straightforward query, like this one:

{
"book_editions_read" : [
  {
    "book_edition" : {
      "id" : null,
      "name" : null,
      "type" : "/book/book_edition"
    },
    "end_date" : null,
    "sort" : "end_date",
    "start_date" : null,
    "type" : "/user/tadhg/tbooks/book_edition_reading_event"
  }
],
“id” : “/guid/9202a8c04000641f8000000004904c64″,
“type” : “/user/tadhg/tbooks/book_edition_reader”
}

Using the Freebase frontend’s view capabilities, I can get a simple list pretty easily.

However, what I really want is something that exposes data about the book (as well as the edition), such as genre and subject, and the author, such as gender and nationality. A bunch of work with Acre got me to this (incomplete) view of books I’ve read this year.

So, it all looked good, and while it’s a little slower than I would like, it’s not the kind of thing I anticipate wanting really quick results from. But, as I was entering more books from this year, I ran into a snag—my data model is flawed as a way to represent the data I capture in my text files.

The reason is that what I consider a “book” isn’t necessarily a physical book (which is what Book Edition maps to), but something more conceptual, a difference clearly exposed when I read a book that’s contained in a collection. A collection such as Viriconium, which contains three M. John Harrison novels and a book of M. John Harrison short stories, and which my text file records represent as:
37. 04/08/2008; The Pastel City; M. John Harrison
108 pages; started 04/08/2008; 1971; [Viriconium; Bantam Spectra/Random House, New York November 2005; ISBN-10: 0553383159; ISBN-13: 9780553383157; FBG: 9202a8c04000641f8000000008dd71df]
38. 05/08/2008; Storm of Wings; M. John Harrison
146 pages; started 05/08/2008; 1980; [Viriconium; Bantam Spectra/Random House, New York November 2005; ISBN-10: 0553383159; ISBN-13: 9780553383157; FBG: 9202a8c04000641f8000000008dd71df]
39. 06/08/2008; In Viriconium; M. John Harrison
86 pages; started 05/08/2008; [Viriconium; Bantam Spectra/Random House, New York November 2005; ISBN-10: 0553383159; ISBN-13: 9780553383157; FBG: 9202a8c04000641f8000000008dd71df]
40. 06/08/2008; Viriconium Nights; M. John Harrison
121 pages; started 06/08/2008; [Viriconium; Bantam Spectra/Random House, New York November 2005; ISBN-10: 0553383159; ISBN-13: 9780553383157; FBG: 9202a8c04000641f8000000008dd71df]

The Book Edition is clearly the Viriconium collection. But if I enter that as a Book Edition Read, there’s no way to denote which of the parts I read, or that there are four books (by my definition) in there. Furthermore, the numbers from my text files and the Freebase data would differ, and that’s clearly not acceptable.

Right now, I only see one way around this, which muddies my data model somewhat: to include Written Work as one of the properties of Book Edition Reading Event, so that I’d be entering both the Book Edition and the Written Work—but could leave Written Work blank if it’s just pointing to the Book that Book Edition points to. Then, I need some special-case code where if it’s not the same as what the Book Edition is pointing to, I go fetch the data about the Written Work instead. It’s just one property, but it makes it harder for other people to use the type (and I’d like to leave that option open), and it definitely increases the code complexity. However, I don’t see another answer that satisfies these self-imposed requirements.

Therefore it’s back to work on this, just at the point where I thought I’d fixed the basic data-entry, data-display, and data-model problems and could have some fun working on graphing out a variety of more-or-less pointless stuff—instead, I have to try to jam the anthology special case into the existing code framework.

Once that’s done, though, there will be graphs, yes there will.

Transcript of the call.
MP3 of the call.
Randall Munroe’s response to the whole thing.

It’s just crazy. A series of people who seem to have difficulty with the concept that 0.002 cents and 0.002 dollars are different. I don’t think they’re pretending to have trouble, either… they seem to actually be unable to comprehend what the customer is saying.

It reminds me another classic, also very close to the truth, that I might have mentioned before: Terrifying Bill Passed During NBA Playoffs.

Its operation is simple: find a person using the FreebaseSuggest box in the upper left, and click on any person to load a page with them at the center. Clicking on the person in the center, or on the [e] links, brings you to that person in Freebase for editing.

The UI needs quite a lot of work, as it looks fairly awful. I might be able to get some help with that this week, and there are certainly a lot of refinements to be made. But as my first work with Acre, I’m happy with it.

While it does show off some of the data in Freebase, it may actually be more useful in showing what data we don’t have in there yet. There are a lot of historical figures whose parentage is widely known but which Freebase doesn’t yet know. Ideally you’d be able to add that kind of thing directly from the app, without having to go to Freebase.com, but that functionality is quite far away…

If you’re curious about building apps, try out the app editor; the source code for my family chart is available, as is code for other Acre applications.

Some potentially interesting other family charts:
Louse XIV of France
JFK
George W. Bush
Elizabeth II of England
James I of England

(The political/royalist bias here is sadly necessary since those appear to be the richest pockets of data…)

Yes, Bacon beat Fries in the the District 14 Colorado State Senate race this year…

Here’s one number that might surprise some people: whites voted for McCain 55% to 43%. (Note: I don’t buy into the suspect concept of “whiteness”, but this kind of breakdown is still significant given how many people categorize themselves, and others, this way.)

Given the amount of self-congratulation I’ve seen in the media, you’d think that white people in this country were far so far in the majority for Obama that racism was clearly dead… but the numbers don’t support that. What they do support, as Chazelle points out, is that white people are no longer a sufficient majority in this country for a relatively narrow majority of them to carry the vote against a candidate sufficiently supported by other groups.

That’s still important. That’s still good news, presenting a lot of opportunities for greater tolerance and less prejudice. But it’s not evidence that white America has suddenly embraced a black liberal, or that (no matter what Obama himself says, incidentally) race is “over” in America.

(One other good sign that must be noted: the only white age group to vote for Obama was the 18-29 range, 54%-44%. Hopefully means the young are more tolerant, and will remain more tolerant.)

“You know, if you were a slave in the old South, what did you get as a slave? You got free room and board, you got free money, and you got rewarded for having children because that was just, you know, tomorrow’s slave. … Can I ask a question? How’s that different from welfare? You get a free house, you get free food, and you get rewarded for having children. Oh, wait a minute, hold on a second. There is a difference: The slave had to work for it.”
—Jim Quinn, The War Room with Quinn & Rose, 6 Nov 2008

I know that idiots like this go on the air just to spew controversial inanities that both outrage others and sit well with their base, but still. (Via Media Matters.)

I’m aware that I’m extremely insulated, living here, and that makes it harder to gauge trends. I can’t think of a single person I know in California who would have supported it. The Bay Area, unsurprisingly, voted against it overwhelmingly. Of all the Bay Area counties, only Solano had a majority in favor. (State-wide, 42 of 58 counties backed it.)

On the other hand, San Francisco’s numbers were 76.5% against, which is certainly a lot, but means that almost one in four San Fransciscans supported it. Apparently much of that support comes from immigrant communities, which is also saddening.

Three legal attempts to prevent the measure from becoming law have already been started. There’s still a chance it won’t make it. I certainly hope it doesn’t, but the fact is that the voice of the people is for it, and that’s really a disappointment.

It’s a big victory for the Mormons, of course; they put over $20 million into supporting it. As Jon Stewart pointed out, this is because the Mormons have always believed that marriage is between a man and… um…

That obvious joke aside, the irony of the Mormons organizing to interfere in the marriage regulations of another state is really quite appalling. Not that I’d expect them to see that.

Apparently the ads that really swung the undecided voters were those that focused on how children would be “forced” to learn about gay marriage; the key piece of one ad was a young girl coming home to her mother and saying (happily) that she’d learned that two princes can marry, or two princesses can marry.

What the problem with that is, I just don’t know, but clearly it strikes some weird fear into the hearts of these people, so maybe one should feel sad for them, as well as for the victims of this measure, whose society is really telling that they’re inferior, and the rest of the state as well, which has been prevented from moving out of that particular mode of benightedness

Although I didn’t vote for Obama, I’m obviously happy that the McCain/Palin ticket aren’t going to end up running the country. In addition, the symbolic importance of Obama’s win is undeniable.

The Democrats are going to have control of the entire government. I wish I could feel happier about that than I do. I’m curious about what excuses they’re going to use for not following a progressive agenda, if they bother with excuses at all.

For the sake of it, here are some things that I’d love to see the Dems do, but which I have very little faith in their actually doing, despite their mandate:

Repeal the Military Commissions Act.
Repudiate the idea that the government can spy on communications between Americans.
Stop mandatory fingerprinting of all foreigners entering the country.
Overhaul the voting system in this country to ensure all voters can exercise their democratic rights without interference.
Reverse the Bush Administration’s drive towards a) greater secrecy and b) executive overreach (e.g. signing statements).

Note that I’m not even including the really big things, like ending (and apologizing for) the invasion of Iraq; enacting stronger consumer bankruptcy rights and protections; drastically cutting war spending and redirecting that money to programs for lower-class Americans; etc., etc. In fact, the five things I listed above are mostly without significant financial consequence and so might actually be doable for Obama, in that his major moneyed backers might not object to them all that strongly. Well, they’d probably object to voter protections, but hey.

Incidentally, Obama supporters should really thank the Pittsburgh Steelers—and note that the fact that they won in 2004 is merely further proof that the election was stolen from Kerry.

Regardless of my qualms, it’s still fantastic to finally see that the Bush period is over, which it would not have been had McCain won, and that McCain and Palin have been kept far from the reins of power. Frankly, as much as I despise the Democrats for mostly just appearing to be a better alternative while doing much of the same stuff as the Republicans, it remains the case that had Obama lost this country might well have been too batshit crazy to have any hope whatsoever. At least with Obama there’s hope from the fact that a lot of people want to go in a better direction.

I know how hard it is to write a novel, never mind a good novel. A series of good novels, that’s certainly a tall order.

Furthermore, Martin’s books are extremely detailed in terms of the world he’s created, the histories of the characters involved, and the intricacies of the political movements. That makes it entirely reasonable for him to take as long as he wants, essentially.

However, what irks me, and what the authors of Finish the Book, George also find objectionable, is the emphasis that Martin places on his other works.

He can write whatever he wants. He’s free to stop here and never finish the series. But that will drive a lot of people nuts, and it’s driving many of us nuts already to read Martin’s blog and discover that he thinks we’re interested in things that are not A Dance with Dragons, or with his working on those things.

It’s a curious phenomenon, really, the expectation created by a series like this. I’m not sure it’s the same as in other forms, such as films, because it’s generally understood that films are so complex and expensive to make that a huge number of people can be responsible for things going wrong. With books, though, it’s just the author. There’s nobody else. There’s nobody to absorb some of the responsibility when things don’t follow the schedule, and I think that fans are aware of that, and get more frustrated because it’s theoretically a manageable situation—just get the one person to write, and that’s it.

Lots of fans understand the difficulty (otherwise, one assumes that they’d write their own books), but the enthusiasm created by a successful series has a flip side, and the flip side is the frustration and anger produced when the creator fucks it up (see Lucas, George) or delays it past the point that seems reasonable—which is what’s happening here.

A Song of Ice and Fire is a fantastic series. I can’t wait for the next book to come out, but that’s been the case for three years now, and it’s hard to maintain the enthusiasm, or even interest, without occasionally veering into the feelings expressed at Finish the Book, George.

On a different note, a fantastic ASoIaF resource can be found at Tower of the Hand. That might help distract you from the absence of the next book… or increase your impatience about its production.

I thought, then, that November would be a good time to do this planning. With a target of sixty thousand words for this novel’s first draft, that’s two sections of planning per day, or maybe ten days of chapter planning followed by twenty days of three sections per day. In any case, it felt reasonable, and I thought I’d prepare myself for it with some consideration and pre-planning.

At around 17:00 this afternoon, the same friend reminded me that November starts tomorrow. Whoops. I haven’t thought about NaNoWriMo, or my editing plan, or my next novel, since that last conversation. It’s still a good idea in theory, but I feel entirely unprepared to start planning tomorrow, or to commit to something like that for the next month.

Nevertheless, it might be a good idea, and might be just what’s needed to get the next novel going. We’ll see.

That doesn’t look like your typical treatment of vampires at all, so I think I’ll have to go see it to see what they’re doing.

For example, I’d like to have the option of paying to be able to watch television feeds of the NFL playoffs, and particularly the Super Bowl; of Yankees playoff games (and perhaps late-season games, depending); of World Cup (and possibly Euro) matches; and of a large number of tennis events: all four Grand Slams, the year-end championships, and to a lesser extent the Masters Series tournaments and possibly some lesser events as well (I’d probably be willing to pay to watch anytime Federer is playing).

Naturally, a certain amount of this content is available online, but both service quality and audiovisual quality can be poor and unpredictable. Traditional television is still far better, and more reliable, as a delivery model—no bandwidth issues, generally higher quality, etc. Eventually I think the Internet versions of these services will get there, but for the moment, they’re not good enough.

What I would really like, if Internet delivery isn’t yet acceptable, is to be able to just pay my cable company for narrow content. I’d be willing to pay $50 for each of the Slams and the TMC, which is $250/year for them right there. I’d probably be willing to pay $30 for each Masters Series event, which is another $270. I’d pay $10 or $15 for the Super Bowl, or $35 or so for the playoffs if I were in the mood for them. And $15 for each Yankees playoff series. That comes to at least $550/year, more if the Yankees are doing well. I’m not guaranteed to do this every year, or for every event, which is problematic for them, but let’s say I’m likely to do it for 70% of the events, so they’d get $385-$600/year, depending.

I’d be willing to pay a monthly fee of $5-$10, too, to “stay connected” to their network, or something, because that seems like it might be reasonable. Let’s say that’s $120. That brings their income from this hypothetical scenario to $505-$720/year.

As it was, they were getting about $600 from me. The answer here is not that I should go back to paying for cable, because many of the events that I want, they don’t have or would force me to pay extra for (mainly the tennis). There must be plenty of people like me, though, who have niche interests.

I suppose the real issue is that most of the people with these interests already pay for television, consider that to be a basic service, and wouldn’t dream of giving it up, and would instead just pay extra. The cable companies are probably afraid that if they made it easy to tailor services, many people would cut their basic subscriptions to the bone. That’s why they’ve resisted a-la-carte for years, and what I want is essentially a version of that, except with more month-to-month flexibility.

I like to think that eventually we’ll get there, though. Eventually, the Internet services might be good enough to do the job, at which point bad things might well happen to the cable companies. Which would be a tragedy, of course.

Sadly, they didn’t make that mistake, and instead it reads:

Only marriage between a man and a woman is valid or recognized in California.

Obviously, this is a completely repugnant proposition: marriage is a legally privileged legal state, and therefore it’s simply unacceptable to discriminate against some people’s access to it.

All the rest of the jumk around this comes from confusion around whether marriage is a religious or legal entity; from rank prejudice against homosexuality; and from inane claims that allowing gay marriage will somehow damage the “social fabric”. Well, that, and a desire to control (indirectly if directly is no longer possible) the sex lives of other people.

There’s a significant chance the Proposition 8 would fail, which would essentially be an endorsement by the state’s voters of gay marriage. That would be a rather major advance, particularly since Proposition 22 passed 61.4% to 38.6% just eight years ago. To reverse that, even by a small margin, a mere eight years later would be rapid progress indeed, and I’d like to read it as part of a general move towards tolerance in this part of the country.

Somehow, I can’t bring myself today to write about something other than a completely ridiculous call to action from some religious nuts. Via Who is IOZ comes this piece of utter insanity, opening of course with a classic spam-like statement:

Dear friends:

THIS IS EXTREMELY SERIOUS.

Minutes ago I spoke with friend Dr. Norman G. Marvin, M.D. and he is so concerned at what he has learned about Barack Obama’s family in Kenya that he is calling a special prayer meeting in his home to pray against the witchcraft curses attempted by them against John McCain and Sarah Palin.
—Block African witchcraft curses against McCain and Palin NOW!, Jim Bramlett, Sep 28 2008

Even without the content per se, I think that there are a bunch of stylistic points that would mark this as extremely dodgy. The first, obvious, giveaway is of course the all-caps line. The second is probably “Dr. Norman G. Marvin, M.D”. Okay, fine, he might be a real doctor. He might even be a real medical doctor with a Ph.D. But its seems common among people who are trying to sell totally implausible crap to insist on impressive credentials. Because, you know, if a doctor, indeed a doctor with doctor both before and after his name, is concerned, then it must be serious. Just like the all-caps bit says. You didn’t argue with the all-caps line, did you? I hope not, because now you’re on the wrong side of the doctor, and that, my friend, is not a good place to be, not at all.

There’s something else, too, that I can’t pin down as precisely, but there’s a “breathless” style here too… “Minutes ago”, for starters. You clearly get the impression that the author, for all intents and purposes some random guy, met with his friend, for all intents and purposes (and despite the highlighted credentials) another random guy, who heard (from yet another random source) something that’s so compelling that he had to… to… well, he had to call a special prayer meeting. In his home!

If we knew the doctor, perhaps that would make this more impressive. Perhaps it’s well known that Dr. Norman G. Marvin, M.D., despite being a staunch Christian who every day heeds the call of Jesus, never, ever, ever holds prayer meetings in his home. Perhaps he never holds prayer meetings at all.

Conversely, maybe he holds prayer meetings at his home all the time, regularly scheduled, so reliably and frequently that they clearly delineate the passing of time for the members of his community, and the unprecedented news is that he’s going to alter his schedule, to depart from the Monday 11:15 discussion on 1 Pet 3:2 to instead pray on this serious matter.

Following that excerpt from Jim’s call to action, the insanity continues, first by reassuring you that next in the chain of sources is Flo Ellers, who is “credentialed with the International Fellowship of Ministries which is based in Washington State” and is also “a member of EndTime Handmaidens and Servants of Jasper, Arkansas”. Well. That puts paid to any doubts we might have had, yes? This person then tells us that the rest of the information comes from “Bree Keyton, a young woman evangelist who had just traveled to Kenya and visited Obama’s home village”.

So Jim heard from Dr. Norman G. Marvin, M.D. that he heard from Flo that she heard from Bree that… wow. I mean, what’s with the intro? Why doesn’t Jim just cite Bree? No, the whole chain has to be called out, apparently.

After that, it’s mainly just paranoid rantings about Obama’s “tribal people” being pagan Muslims… yes, Muslims who pray to genies, apparently. I somehow don’t see how that fits in with Islam. Incidentally, the Satanic pagan Muslim of the piece, Ralia Odinga, himself claims to be an Anglican, but maybe that’s not so far from Islamic paganist Satanism in the eyes of people like Jim, Dr. Norman G. Marvin, M.D., Flo, and Bree.

Some final gems. First, after all of that: “Dick Morris of Fox News was sent to Kenya to help Odinga run his campaign! I find that unbelievable.” Yes, clearly that’s the most unbelievable part of the entire thing.

And lastly, this really has to be the ultimate in self-delusional defensive thinking:

The occultists are “weaving lazy 8’s around McCain’s mind to make him look confused and like an idiot”.