On Wednesday, I posted my three questions about the recent Radicati Market Analysis. These questions were my own, and arose after I read the Radicati Market Analysis and the response from Michael Sampson, of Shared-Spaces Research and Consulting. As a technologist, I read both papers and I felt that the questions that I had were reasonable and that they were ones that any reader should know the answers to before making an IT purchasing or planning decision based upon the Radicati (or anyone else's) market research projections.
Here were my original questions:
1. Who is the actual author of the report?At the conclusion of my post, I publicly solicited answers to these questions from anyone who might have been involved in the production of this particular market analysis, who may be able to shed some light on my questions. I also invited Dr. Radicati, who I assumed authored (or at least approved) the research, to personally respond to my questions, and I offered to post her response in its entirety on my blog.
2. Who paid for it?
3. What were the sources of information used and how did they lead to the conclusions presented in the research paper? I asked if it would be possible to see a list of citations of the sources consulted and for the figures and graphs presented?
I have not heard anything yet; however, this evening, I noticed that Dr. Radicati had posted an official response on her Web Site entitled, "Our Response to Ed Brill's Weblog."
Let's see if Dr. Radicati answered any of my questions:
In her response, Dr. Radicati states that the views contained within it represent the "company position," and that any questions may be directed to Sara Radicati directly. OK, so that answers question number one. Sort of. Continuing to question number two, I read that this paper was not funded by any particular vendor and that it is "based on excerpts from the 5 full length reports..." and it goes on to list reports which may be purchased for $3,000 each. OK, that works for me. If I had five papers to sell, I too, would likely offer a summary paper, showing highlights of my best work, free of charge, so that the public might get a glimpse of the quality of my work and hopefully, want to buy my full length reports. As far as I am concerned, I will accept that as an answer to my question number two. This leaves me with only my question number three.
Before I go on, I would like to publicly state that I do not work for IBM or Microsoft; however, I represent many clients, who make sizeable investments and purchases from both companies each year. As an independent consultant for ICA.COM, Inc., I consult on the technology integration of products from these manufacturers and others with an eye towards making technology work to increase productivity. Internally, I use Microsoft Windows 2000, XP, and Office, as well as Lotus Notes for all of my Information, Communication, and Action tracking. Oh, I also use Outlook and I have corporate clients who use Lotus Notes, Outlook, and Exchange, for their messaging platforms. As you might expect, I am interested to know what others have to say concerning the future of any of these products. I am also very interested to know how they reached these conclusions, so that I may learn from them and advise my clients accordingly.
That said, I now come to my question number three -- the one that was of greatest interest to me as a technologist: "What were the sources of information used and how did they lead to the conclusions presented in the research paper?" No response. Michael Sampson, a consultant and analyst in messaging and collaborative technologies, made accusations of poor quality research and analysis when he wrote:
My overall analysis of this White Paper is that it is a headline grabbing publication lacking analytical rigor, logic and appropriate follow-through. The paper is unbelievable in this respect, making me wonder whether it is really an independent publication, or marketing material sponsored by Microsoft. The author seeks to compare two platforms that are entirely different in terms of scope and imagination, and then recommends a path for clients. I strongly believe that the author entirely misses the point, is totally wrong in the commentary on Microsoft's messaging strategy, and therefore provides market share growth figures that are just plain wrong.Well, Michael was certainly right about the headline grabbing part, and I suspect that he's probably right about the rest, too. I note that Michael did not just make his bold accusations without supporting them. In fact, He wrote a paper, complete with citations, in response: Response to the Radicati market analysis study of June, 2004. In it, he responds, point by point, to the information and projections by Dr. Radicati. (I encourage you to read both Michael's response and the original Radicati paper for yourself and formulate your own opinion. I'm curious to know what you think about either paper.)
So far, there has been no response from the Radicati group or its president, Dr. Radicati, publicly or privately. (I checked with Michael this afternoon.) This makes me wonder: why did Dr. Radicati not respond to Michael's detailed response? Could it be that he was right?
Concurrent with all of this, there has been a lot of excitement over at Ed Brill's blog, here, and here, with each generating pages of comments. (You can draw your own conclusions about this.) The issues going on at Ed Brill's web site, in my opinion, are small, compared to the alleged fundamental flaws in the Radicati research, as pointed out by Michael Sampson. If this were a political campaign, I might wonder if the things that happened on the Ed Brill site, were intended to distract our attention from the real issue...
The Radicati response continues in Q2:
Essentially, none of the content of this whitepaper is new - the paper is entirely based on information which we have already published and market numbers which we have already made available to the press over the last 7 months in countless interviews, articles and discussions.OK, so are my enterprise clients and I to assume that the "analytical rigor, logic and follow-through" demonstrated in this "free" paper by the Radicati Group and promoted by Microsoft is representative of what we would find if we paid $15,000 to buy the full length reports from which it was "lifted?" If so, what kind of impression should we then formulate about the suite of Radicati research papers? What should we now think about Microsoft or other vendors when they quote these reports on their web sites? Who should we trust?
In the closing of Sara Radicati's response to Ed Brill, she writes:
"Finally, we believe that the comments on Ed Brill's blog represent his own personal opinion and that of his friends, and do not reflect the opinion of IBM Lotus' management."
I am not going to get involved in the discussion about what transpired on Ed's site; however, I have been following it with amazement. I am interested to see what IBM and Microsoft have to say. As for me, I will simply close this lengthy post by asking one last question of Dr. Radicati:
Dr. Radicati, will you post a public response to Michael Sampson's response to your paper, or should I assume, that Michael's observations and responses are correct? My clients and I would like to know the answer to this question. A few others may be interested, too.
As before, I will close with this offer: Dr. Radicati, if you would like to respond directly, and I hope that you will, you may reach me at my email (above) or at my office, 661-242-8410 x101. I look forward to hearing from you. I promise to post any written response from you in it's entirety.
Eric Mack.
Discussion/Comments (7):
I guess I missed this earlier: The Radicati report is free, presumably as a PR tool. OK, I'm fine with that. But if it is truly "free," then why should Microsoft have to license it to share it on their web site? Would I have to license it if I wanted to share it here as a service to readers of my blog? What if my blog were mentioned favorably in the report? Would I have to pay more?
If it was decided that the report was to be free because The Radicati Group wanted to use it as a PR tool, then I should think that they would be flattered that anyone would want to post it on their web site and that they would welcome the opportunity, not charge for it.
Perhaps I just don't understand the way the system works.
If I wanted to post their free paper on my site could I do so without paying a license fee? What's the difference? Other than the fact that I do not have a product that is glowingly mentioned in the paper, I cannot think of any.
Given that the other Radicati reports cost $3,000 each, I would expect that the license fee that Microsoft paid in order to "Share" the free Radicati report is not an insignificant amount.
I have no problem with The Radicati group licensing a paper to a company to put on its own site. However, if I am to believe that a paper is truly free and that there are no undisclosed motives for it being shared without cost -- other than perhaps PR -- then I would expect that it would be just that - free.
I know, a possible response might be: "but it is free -- to the customer." But that begs the question about it being free after all. If Microsoft, IBM, or anyone else has to license the report from Radicati to share it -- for the convenience of its customers, of course -- is the paper really free? Might that create a profit incentive to write glowing free papers to license? I'm just curious.
I guess what this really boils down to, for me, is the issue of undisclosed interests. If the response had said, "We were paid a lot of money to write ad copy that looked like a marketing report," I would have dropped the issue. Magazines and Newspapers do this all the time. But when my clients and I are asked to believe that a paper represents objective research and analysis, then I expect a little more professionalism.
It's late. I never intended to get wrapped up in this topic; it has taken more of my time than I expected. I did learn an important lesson from this experience:
I shall be much more careful when I review or recommend market research reports for my clients. In fact, I will probably never look at a research paper (from any organization) quite the same way again.
Am I off base in asking these questions or expecting professional conduct in this area?
What do you think? I'd like to know.
Eric
http://www.ica.com
Posted at 07/31/2004 2:09:42 by Eric Mack
Just wanted to offer a possible suggestion about why Dr. R isn't responding - it is entirely possible that since she has not blogged or commented on blogs before (at least I have not seen such activity), that she is refraining from doing so now? From the company's "response", it appears she DOES indeed welcome direct queries, so perhaps if you copied and pasted your relevant blog entries and comments and sent them to her, she will respond? Then perhaps if you obtain her permission, you can share these with us? I am really quite interested to hear the response.
Just a suggestion, from a reader who is looking for some "closure". :-)Good luck!
Posted at 07/31/2004 19:46:52 by Daniel Johnson
"If this were a political campaign, I might wonder if the things that happened on the Ed Brill site, were intended to distract our attention from the real issue.."
I doubt that. I think it's been clear that it was a sideshow, but I doubt deliberately so. Who goes and cleans up their personal weblog at 10 PM local time only after it's been discovered? No, I think that whole bit was a comedy of unethical errors.
The attention will be focused back on the real issue during the coming week -- with the publication of IBM's formal response (hoping it comes on Monday, but could be Tuesday) and at least one press article on the whole thing.
Posted at 08/01/2004 10:56:49 by Ed Brill
Daniel, you bring up a good point. I assumed that Dr. Radicati was aware of my blog, at least from the surge in traffic re: Radicati. I took your advice and sent her an e-mail this evening. Here is the text of that e-mail.
From: Eric Mack
Date: 08/01/2004 10:49 PM
To: sara@radicati.com
cc:
Fax to:
Subject: Dr. Radicati, do you plan to respond to Michael Sampson's response to your Messaging Market Analysis?
Dr. Radicati,
My name is Eric Mack; I shall assume that you are aware of me -- if not for work with the EMA many years ago, when I was CTO of Peloria Technology Corp, then at least from my recent comments on my personal blog: www.EricMackOnline.com.
Michael Sampson's response to your Messaging Market Analysis a week and a half ago raised many questions for me and for my clients. Many companies and trade publications refer to your reports and analyses for information on the messaging marketplace. As you can imagine, Michael's response has generated more questions than it answered.
Your recent response to Ed Brill's blog on your web site answered several of the questions that I had about motivation for the publication of the report, thank you. What I was hoping that you might also address, were the specific issues that Michael Sampson raised. So far, I have seen no response to any of Michael Sampson's comments about your analysis -- from you or from anyone else. Michael's objections to your analysis and conclusions were specific and clear. I'd like to know what you think.
Dr. Radicati, will you post a public response to Michael Sampson's response to your paper, or should I assume, that Michael's observations and responses are correct?
If the referrer activity on my web site coming from other blogs and from Google searches seeking "Radicati" is any indication of interest in this topic, I am not the only one with questions. I know that at least my clients and I would like to know what your response is to the Sampson paper.
As I have offered on my public web site, if you would like to respond directly, and I hope that you will, you may reach me at my email (below) or at my office, 661-242-8410 x101. I look forward to hearing from you. I promise to post any written response from you in it's entirety.
Eric Mack.
----------------------------------------------------
Eric Mack,
eProductivity Specialist,
Making Technology Work For You
VideoPhone: 661-665-7878 H.320 Codec, 2x64 (ISDN)
Blog: http://www.ericmackonline.com
Corp Web Site: http://www.ica.com
eProductivity http://www.eProductivity.NET
WeatherCam: http://weather.ica.com
Homeschool: http://www.mackacademy.com
Facsimile: 661-242-0171
Telephone: 661-242-8410 x101
--- Nothing new below / Fin du message transmis ---
Posted at 08/01/2004 22:54:19 by Eric Mack
I just received a call from Dr. Radicati's assistant, who informed me that Dr. Radicati received my email forwarded it to her to call me to let me know that Dr. Radicati was away for a few days and that she would respond to my e-mail on Wednesday.
Posted at 08/02/2004 9:42:13 by Eric Mack
Eric, this is a great summary and critique to this debate and a great example of the power of blogging. Analysts are going to think twice next time they show such obvious bias in the future. I blogged a little on this topic as well as I was just to appalled not to make some comment:
http://radio.weblogs.com/0135175/2004/08/04.html#a198
Posted at 08/04/2004 4:37:10 by Steve
so I am reading this and what was the response from her e-mail on Wednesday?
Posted at 09/19/2005 12:55:18 by ..and?
Discussion for this entry is now closed.