Ad hoc text analytics

I found an old sentiment analysis application. It has very unglamorous packaging but a  good algorithm under the hood. I ran the Twitter user id’s of the brightest people I know. well, know of, who are active Twitter users. The assessment of “bright” was subjective by me.  All are acknowledged experts or advanced degree holders. Maybe half speak English as a second language, but are sufficiently articulate that their “essence”, well, intelligence shines through.

Guess what: It worked! I don’t know if anyone cares about this sort of thing, that really sharp successful people score well on this sentiment analysis indicator. That doesn’t necessarily mean it would have any predictive value. And no one seems to care much about this anyway. But what I’m saying is that most of these people only have okay-ish Klout scores e.g. 40′s. But they’re not trying to use Twitter for any particular social media purpose. Well, I don’t know that with certainty.

Monetize Your Followers

As a statistician and mathematical modeling practitioner, I’m not a stranger to the concept of quantifying the value of intangibles.  In the ethical framework in which I studied and worked, such quantification might be applied to a concept such as negative dollar value of ill-will (per person) generated by denied boarding due to passenger aircraft overbooking.  Yet I found myself rather unnerved today by TweetUp’s article “How much is a follower worth?”.  According to TweetUp analytics, the answer is $136.80, as of June 2010.

What is so troubling about this? I believe in free enterprise, von Neumann economic utility theory, liquidity, efficient capital markets, Keynesian economics,  freedom of choice for retail consumers, Adam Smith, “survival of the fittest’  (more or less), evolution, Lord G-d in Heaven, free will, democracy and the American Way. However, I suppose I draw a distinction between the categories of “friends and family” versus “customers and clients”. I want to look out for the best interests of the first category. However, I’m willing to let the Efficient Frontier do its best for the rest, knowing that in the long-run, everything should work out.

Friend, Fan, Follower or Something Else Entirely?

Twitter assigns two only two types of user roles: follower and following. Google’s Picasa photo application assigns roles of friend, fan and favorite, which is no less confusing. Google Buzz is more flexible, allowing users to name categories of contacts without restriction. One may have family, friends, colleagues/ coworkers, business contacts and so on. Facebook is starting to move in the right direction, with friends kept distinct from fans, the latter usually pertaining to organizations or public figures. However, there remains no ability to capture the nuances within the friend category.

I perceive these oddly and inconsistently defined user roles as a discontinuity embedded within the social networking model of Web 2.0. For many months, I could not articulate my unease with certain social networking applications, but not others. Now I understand. I balk at the logical inconsistency and behavioral proscription (as defined by my personal moral framework) in these follower/friend (but no family) role definitions.

My instinct is to avoid any entangling of ties between family/ friend/ loved ones and my customer base. Behavior that is required to achieve the goals of commercial enterprise is not appropriate for those with whom I am emotionally involved.

Perhaps the conflict can be resolved through better nomenclature.  I have a solidly codified standard of behavior and obligation toward those I call Friend. I do not wish to monetize my friends. That seems exploitive and morally wrong.  The language of business is entirely different. I see no moral hazard* in valuing one’s customer base.

This is my call to action: We must use better terminology! Linkages capture non-numeric relationships. Today’s internet is the world of the semantic web. Text analytics. The Social Graph. URL’s and URI’s. FOAF (“Friend-of-a-Friend”).

Mark-up language is not just for hypertext anymore! Nor is it confined to the extensible (XML) variety. The Worldwide Web Consortium (W3C) lists no fewer than 50 common and uncommon industry and cross-industry entries in its schema library documentation. And even this list is in a continuing state of flux as it evolves to meet the needs of users.The schema document includes

Biology, Chemistry, Geology, Weather, Mathematics

• BioML Biopolymer Markup Language
• CML   Chemical Markup Language
• XMML  eXploration and Mining Markup Language
• DWML Digital Weather Markup Language
• MathML  Mathematics Markup Language

Business, Finance, Taxation, Real Estate

• B2MML  Business to Manufacturing Markup Language
• FIXML  Financial Information eXchange Markup Language
• Tax XML  Federation of Tax Administrators
• RETS  Real Estate Transaction Standard

Government, Legal, Education

• EDXL  Emergency Data Language Exchange
• JXDM  Global Justice XML Data
• SIF XML  Schools Interoperability Framework XML

and many more including healthcare, publishing, pharmaceuticals, media, software and even metadata. Most have yet to be implemented. Yet many will eventually come into popular use: MathML 2.0 was comprehensively defined, tested and documented by 2003, but was quietly sidelined until the time was right.  That time happens to be now, as it is one of the features supported by HTML5.

In fact, WordPress, the digital publishing platform I am using to compose this bit of truculence is fully enabled for the Friend of a Friend (FOAF) project whose ultimate goal is creating  a Web of machine-readable home pages describing people, the links between them and the things they create and do.

We are defined by our use of language. My neighbor, my mother and my students need to be classified differently than my current or potential customers. It is morally repugnant to apply valuation methodologies to those we love and care for. We are in the early stages. Let’s build the framework correctly and standardize our usage of “Friends”, “Followers”, “Favorites”, “Fans” and “Subscribers”.

Text analytics application du jour

The Visual Thesaurus is worth a glance for amateur etymologists as well as writers. Linked node data visualization transforms the elderly thesaurus into a much more useful reference tool. Usage is fee-based but nominal.

Thesauruses (thesaurii?) identify context-appropriate and idiomatically correct synonyms.  Apparently this isn’t so easy to for a search engine, Google included.  A recent article from Wired.com went into a lot of detail about Google’s word search algorithms, and made the amusing disclosure that distinguishing between  a “hot dog” and a “boiled puppy” is still an issue.

Wordle Finally Arrives…

I’ve found that works by T.S. Eliot are particularly suitable for wordl-ing.  This one is an excerpt from the Nobel Prize-winning poet’s The Love Song of J.Alfred Prufrock.  In order to view in full splendor, clicking on the image is unfortunate but necessary. as Wordle is covered by some rather specific licensing agreements. 

Let us go then, you and I...