Poker and Artificial Intelligence

Today, I was able to go to a talk by Michael Bowling from the University of Alberta.  He was visiting the University of Denver, and he spoke to a crowd of CS and other science types in Olin Hall. 

He did a good job of explaining how the game of poker can be used in the course of programming artificial intelligence into computing systems.  This paper (Poker as a Testbed for Machine Intelligence Research PDF) from his lab in 1998 covers some of the basics.

He showed a good clip from the movie Rounders, and then he explained some of the basic rules of the game and some of the strategy involved.  (I still can't believe that Matt Damon didn't consider that KGB might have had pocket aces during their initial game in the movie....) Poker is more like real life for a number of reasons.  You have to deal with incomplete knowledge of the game (you can't see the cards dealt to your opponent), and the opponent can try to bluff his or her way out of a jam.  This is not the situation for chess and other types of board games.

Anyway, I learned that it now possible for computers to beat world class professional poker players on a head to head basis.  They programmed the Polaris system to have different personalities, to be agressive or passive or bluff sometimes or whatnot.  The Alberta folk will continue to work on poker games that involve three or more players.  The programming gets a bit more complicated with more players.  Polaris was programmed to work with Texas limit hold'em so that the raises were set at particular levels.  There are just too many raise variables to consider in no limit hold'em, but they could work on this for future projects. 

Other links to check out.

• Wired article - "Ante Up, Human: The Adventures of Polaris, the Poker-Playing Robot"
• The Second Man-Machine Poker Competition - http://poker.cs.ualberta.ca/man-machine/
• At the Twenty-Sixth Conference on Artificial Intelligence (AAAI-12) in Toronto, Ontario, Canada there will be the Seventh Annual Computer Poker Competition -  http://www.computerpokercompetition.org/ 
• Nash is still cited in their papers. Nash, J.F. 1950. Equilibrium points in n-person games.
  National Academy of Sciences of the USA. 
• Their various publications, http://poker.cs.ualberta.ca/publications.html

Wrapping my brain around two posts, should we step in the way so we avoid the bullet?

I've got two blog posts to think about, and I am not quite sure how they support each other (or contradict each other) but I find both to be very interesting.

One is "Engineers Crashing Our Gates" which is a follow up of the Tennant post "You Never See the Bullet That Takes You Down" (found via Jason).  Essentially, companies and organizations do not see what hit them.  Did Blockbuster not see NetFlix?  Did the Travel Agent Industry not see the Internet coming?  Does Elsevier not see Open Access and Open Science coming? What of Librarianship?

Then, there is this post from Dave Puplett on "Academics must be applauded for making a stand by boycotting Elsevier. It’s time for librarians to join the conversation on the future of dissemination, but not join the boycott."  He notes in the comments section that "I [Dave] really do think Librarians have a huge role in advocating change in Scholarly Communication – please see a previous post of mine here: http://blogs.lse.ac.uk/impactofsocialsciences/2011/10/28/championing-open-access/, or some of my other work: http://scholar.google.co.uk/citations?user=VmFORfgAAAAJ&hl=en ....  My point is that if the Academic community is ready to use its voice to lead on this issue, it’s time for Librarians (who have been agitating for a long-time here) to joint the choir and not get in the way."

This logic just seems backwards.  Librarians are in the center of this scholarly communication battle, and we had better try to see what bullets are coming our way.  We need to have the best vision possible, hone our karate skills, and try to predict what the others are going to do to attack us.  For librarians to step to the side of the ring and let only the scholars duke it out with the publishers while we watch from the 17th row is plain stupid.  We need to be in the ring fighting for our patrons, fighting for what we do to serve humanity, and fighting to preserve culture and the record of knowledge.

The scholarly community should have a loud voice on scholarly communication issues (Duh), but they can often be focused solely on the authors and researchers and their interests.  The Library community has the interests of readers and undergraduates and staff and other information consumers at heart.  We are there to speak for them.  The librarians and our patrons should not be told to "not get in the way."  In short, we need to insert ourselves so that we can get in the way.  The more we get in the way of publishers and scholars, the less we are in the way of that bullet.

The clash of two cultures: Scientists, academics, scholars vs commercial publishing and Elsevier #RWA

I am not sure if this is the main root of the problem between scientists and some commercial journal publishers, but it seems to be a problem between the non-profit altruistic mindset vs the profit motive mindset.

For the most part, scientists and scholars want to publish articles to gain scholarly recognition, academic rewards such as promotion and tenure, and for the prestige of publishing in a high impact factor journal because they only accept 10-15% of papers, etc.  They are researching and writing for altruistic reasons such as advancing science, make the world a better place, helping others learn how the world works, etc.

Back in the 1950s and 1960s, scientists started using many more commercial publishers because there was a lot more research being done and they needed outlets since society presses were not able to absorb the large increase of article submissions.  The commercial presses learned that libraries generally didn't cancel journal subscriptions because of institutional momentum, and an inelastic market for the articles existed.  (Academic libraries keep subscribing because the patrons keep on demanding access.  Patrons demand access even though they are not the ones paying for the content.)  So, publishers such as Elsevier learned that scholars publish and review articles and edit the journals for next to nothing because they want recognition, not money.  So, the publisher takes advantage of the situation, raises subscription rates because libraries want to please their patrons, so they don't cancel very often.

20-30-40 years ago, publishing information to hundreds or thousands of others was hard, printing and mailing was difficult, laying out articles was tough--particularly scientific content.

Then the Internet happened.

Scientists are smart and have figured out that global publishing is easier than commercial publishers let on.  They are starting to take advantage of new OA outlets such as PLoS and others.

But, the culture of scientific publication had been ingrained in scientists over hundreds of years, and it takes a while for some scientists to change their perspective.  They still want to get noticed, get scholarly rewards, get tenure, and get cited by others.  So, they have lists of journals that junior scholars are supposed to publish in.  However, some of the more forward thinking scientists understand that publishing in OA sources will increase their reach, increase their citations, and increase their standing in the academic community.  (The Wellcome Trust says “it is the intrinsic merit of the work, and not the title of the journal in which an author’s work is published, that should be considered...") They do not need to submit their articles to closed-access (toll-access) journals.  There is a reason that PLoS One is the largest journal in the world with a solid impact.

The profit motive of some scientific publishers has skewed their perspective.  They do not really care about maximizing the reach of scientific knowledge.  They want to hide the articles and the research, because they are used to an age of information scarcity, but we no longer live in that time.  They are trying their best to hold on, and it shows.  They may claim that they support universal access or something like that with a sustainable business model, but that is code for--"We want to keep on sucking more and more money from libraries and subscribers for as long as possible. We like our 36% and 42% profit margins, thank you very much. We will provide access to a small segment of the developing world because it makes us look good, and we don't get much of an increase to our server load--all we have to do is turn on some software switches to some IP addresses."

I know that there are other variables and opinions, but this seems to be the crux of the problem.  Most scientists don't care about making a profit off the research, and they want to be cited and read by as many people as possible.  The commercial publishers that used to be seen as supporting the scholarly communication interests of scholars have diverged too much.  They want to lock away research and knowledge.  The two groups are diverging, and the scientists no longer need to support the aims of the for-profit commercial publishers.

Yet another great review of Reinventing Discovery

The book, Reinventing Discovery, is amazing!  Go out and buy yourself a copy RIGHT NOW!  Really, it is that good.

Considering how much this book covers the issues and problems of the modern scholarly communication industry, I was surprised to see that the term libraries only shows up once in the index of the book.  Even then, libraries are only mentioned in the notes section of the book.  He notes that academic libraries and publishers are not institutions that will be developing the new tools for the construction of knowledge in the future.  He says that "the right place for such tools to originate is with scientists themselves." He also noted that "libraries and scientific publishers are not, for the most part, set up to work on such risky and radical innovations."  For the most part, I agree with the statement that many librarians and library systems are risk averse, but there are many librarians (and some publishers) who have great ideas, insights and will provide innovative solutions for the long term advancement of scientific communication.

I know that Michael received a lot of advice and communication from members of the library and open science community, so I am a little surprised that libraries were not mentioned more often in the book.

The best chapter for me was Chapter 9: The Open Science Imperative.  Much of the book and much of the chapter is available online at Google Books, so go read some of it there.

I wish Michael would have talked a little more about some of the cultural issues that are holding scientists back from doing more open science. There are some positive financial and social incentives to encourage more scientists to do open science, but academic department administrators and deans still tell junior faculty that they have to publish in the closed access Journal of X, the limited access Journal of Y and/or the Transactions of Z in order to get tenure.  The academic bean counters may find it easier to evaluate a researcher based on the supposed quality of journal title containers, than it is to evaluate the quality of specific articles within those journals.  Mike Taylor noted that:

Because of the stupid way researchers are usually evaluated (and this is another whole issue), the intrinsic quality of our work matters less than the brand name of the journal it’s published in. So we have strong selfish reasons for wanting to get our work into the “best” journals, even if it is at the cost of effective communication.

and

Happily, there are signs of movement in this direction: for example, The Wellcome Trust says “it is the intrinsic merit of the work, and not the title of the journal in which an author’s work is published, that should be considered in making funding decisions.” We need more funding and hiring bodies to make such declarations. Only then will researchers will be free of the need (real or apparent) to prop up parasitic publishers by sending their best work to big-name, barrier-based journals.

More change needs to take place within the hundreds of academic departments.  But, if the major funders such as the NIH and the Wellcome Trust (and many other funding organizations) can continue to provide money as the carrot for positive open science innovation, then we have a bright future ahead.  This is why I support the Federal Research Public Access Act. When this passes, this will encourage more scientists to do science in the open.

Encouraging more discussion about open science #rwa

I posed this question to people in an internal Elsevier forum. I wonder what kinds of responses I will get?

Concerning the recent Elsevier boycott from a number of scientists, a representative from Elsevier Science noted that they "need to do a better job of communicating" with their readers, subscribers, authors, librarians and other interested parties.

Michael Nielsen, author of Reinventing Discovery [Note, I will finish the book tonight], recently said that "we need positive actions as well, not just an agreement about what not to do! But what’s good about the boycott is that it has engaged lots of new people in serious discussion about better ways of doing and communicating science, and some of those people are taking action. That’s exactly what’s needed for open science to thrive." Open Science is coming. When Elsevier supports legislation like the RWA, that is not going to endear many researchers. My question is, what do you recommend Elsevier read to understand the cultural shift that is happening in open science?

My #libday8 post

Here is my LibDay8 post.

My day started with a 9am once-a-month PIRLS meeting.  (PIRLS is short for the Professional Information and Reference Library Services Team).  Some of the stuff we went over was: Brainstorm programs to accompany the October 3, 2012 Presidential Debate; in the renovated building, what should we call the "Access Services Desk" [Not that]; look at new mockup of ILL webpages and GIST; Law catalog merger issues and fixes; Genre now listed in the catalog; Policy Council and other team reports.

The meeting broke at about 10:15am, and then I was able to attack my email.  I only had about 30 emails that had come in over the last 16 hours.  (I usually don't check my email at home, since I want to deal with work problems while I am actually at work.)

Some of the things that I dealt with were:

• A question about the SLA All-Sciences Poster Paper Session. (An official announcement will go out tomorrow.)
• An ILL popout screen does not work properly in IE (but fine in Firefox and Chrome), so I reported it to the relevant fixit people.
• I started to draft out an email as a reminder to DU LIS students about the RMSLA Shirley Alldredge Lecture and Annual Business Meeting, but someone else beat me to it. I am the SLA DU Student chapter adviser.
• Forwarded information about the SLA Engineering Division $1200 SPIE DigitalLibrary Student Travel Stipend Award.
• One of our staff wrote a new blog post on the Academic Commons page, so I posted it to our Twitter, Facebook and our Google+ accounts. I finally set up the Academic Commons blog on TwitterFeed so it would autopost to Twitter.  I had been meaning to do that. I also FBed and G+ed some other items from our other blogs, the Penrose Pen and Good Answers.
• I am going to be speaking to science undergraduate students on the evening of February 15th (free pizza!), so I looked at a draft of a flyer that is going to be distributed to advertise the event. Looked fine to me.
• Sent out a draft of a future blog post on the Penrose Pen concerning the book display for February.  It will have a focus on "African Americans and the West."  Another Reference faculty member agreed to post it on Monday the 6th. [January is here.]
• Agreed to meet with the President of the Rocky Mt Chapter of SLA for a chat. (Hi Rachel!)  Recommended that some of the students in the SLA Student Chapter and some others from RMSLA should join us. She will send us a Doodle.
• My Chemistry Department Liaison had not heard of the Elsevier flap going on, so I sent him a mess of links concerning the issue.  I told him -- There are many, many news items. Here are some of the major ones in just the last week or so. 
• http://lj.libraryjournal.com/2012/01/publishing/petition-targeting-elseviers-business-practices-begins-to-snowball/ 
• http://www.economist.com/node/21545974 (The price of information - Academics are starting to boycott a big publisher of journals) 
• http://poynder.blogspot.com/2012/01/elsevier-needs-to-get-out-more.html 
• http://blogs.library.duke.edu/scholcomm/2012/01/31/why-boycott-elsevier/ 
• http://www.techdirt.com/articles/20120130/13030217589/will-academics-boycott-elsevier-be-tipping-point-open-access-another-embarrassing-flop.shtml 
• http://chronicle.com/blogs/wiredcampus/elsevier-publishing-boycott-gathers-steam-among-academics/35216
• http://www.forbes.com/sites/timworstall/2012/01/28/elseviers-publishing-model-might-be-about-to-go-up-in-smoke/ 
• http://the-scientist.com/2012/01/18/cracks-form-in-anti-open-access-push/ 
• http://www.wired.com/wiredscience/2012/01/testify-the-open-science-movement-catches-fire/ 
• For a list of posts done before January 17th, see http://scienceblogs.com/confessions/2012/01/around_the_web_some_posts_on_t_1.php  
• I heard back from the Chemistry Professor, and he noted some economic concerns in favor of the commercial publisher.
• I contacted a prominent scientist in the UK about arranging a meeting this fall for my sabbatical on scholarly communication issues, but I have not heard back from him yet.
• Chatted with one of our acquisition staff about two journals (One I ordered and one was supposed ot be cancelled, but it looks like we won't be able to get the new journal, because the new publisher won't let us cancel the old journal.  Long story. 
• Sent out this announcement about the Executive Director opening at the Alliance. Are you interested?
• Wrote this blog post concerning LibDay8.

I still have 34 minutes left to my day here, so I will try to spend a little time working on my article concerning OA resources that are used in Africa, maybe clean up my desk a little.

We are going to have a major snow storm tomorrow, so I think I will take my laptop home with me in case tomorrow is a work-from-home day. I am supposed to come in for a monthly Penrose Library Faculty Meeting, but that might be cancelled.  We will see.

Another #RWA response from me. More to come, I am sure.

This is a comment I left at another blog.  Thought it was worth its own blog post.

Nobody is telling Elsevier that they have to accept manuscripts that had been done with NIH or other US Federal funding. They could just say, “Dude, take your article elsewhere.” It isn’t like they don’t know that the author final manuscript is going to the PMC 12 months later.

Let me be clear — the versions of articles that are deposited with PubMed Central are manuscripts that have not been peer-reviewed. Here is a list of journals that do/do not accept the publishers “work product.” http://www.urmc.rochester.edu/hslt/miner/research_and_publishing/publisherspoliciesonpubmedcentralminerlibrary.cfm
 
If anything, the actions and public letters from Elsevier, Wiley, AAA, etc. (See at http://www.whitehouse.gov/administration/eop/ostp/library/publicaccess) will drive more scientist manuscripts to OA publishers. This way, the public will get to easily read the final published peer-reviewed version instead of only having free access to the author manuscript.

The scientists are catching on, the publishers are afraid of losing their cash cow.

A week after #scio12, #rwa and reinventing discovery

This week has seen a convergence of three topics.  First, I got back from the Science Online 2012 unconference a little over a week ago (1/22).  I had been meaning to write a post to wrap up the sessions I attended, but then I started to see a lot of news concerning the Research Works Act.  I have also been reading Michael Nielsen's Reinventing Discovery, and there has been a good deal of discussion about that book on the net as well, including from the famous #scio12 @BoraZ.  So, with all this open access and open science discussion swirling in my head, I figured it was a good time to put the electrons down on the blog.

1. Science Online 2012.  I tweeted a bunch of the sessions already (and blogged about one), and most of the sessions have some form of online abstract, so I don't need to go into the details.  What struck me most about the conference was the discussion between the science journalists and the scientists themselves.  Scientists are stuck in a hard place because many academic departments and/or institutions frown on bloggers/tweets and people who try communicate their research to a general audience.  Journalists have a hard time working with scientists who do not understand their craft.  These two sessions particularly caught my attention.
2. Research Works Act (RWA). While I had known about the RWA since well before the unconference (January 5th), the topic didn't catch fire with scientists until a post from Fields Medalist, Dr. Gowers wrote "Elsevier — my part in its downfall".  That sparked a huge amount of discussion and other blog posts from a variety of scientists.  Many of those posts are cataloged at Michael Nielsen's Polymath Wiki page on journal publishing reform. (I see some posts that are missing on the wiki, so I will add those later.)
3. This brings me to Michael Nielsen's book Reinventing Discovery.  I have been slowly reading the book (a library copy), and I was reading it on the way to and from the Science Online Conference.  Two copies were being given away at the conference, but alas, I didn't win a copy.  In any case, here are some good reviews of the book by Bora, John Dupuis and Martin Fenner.  Michael also talked about his book on Science Friday.  Here is the podcast last Friday, January 27th, bit.ly/wFibOk.

Levels of copyright and some questions

Amy, Jenica and Andy brought up some good questions concerning copyright, and the copying of music and audio files from library materials, and should librarians talk to (confront?) a patron concerning actions that could be deemed to be illegal. But, there are a whole range of actions that patrons could do with library materials (either in the library or with the materials checked out).  At what point should I mention something about copyright to the patron?

1. Patron checks out a CD from the library.
2. Patron checks out a music CD, and tells the circulation person that the CD will be returned tomorrow because the person is going to rip the music to their iTunes library.
3. Patron checks out 20 music CDs, and tells the circulation person that the CDs will be returned tomorrow because the person is going to rip the music to their iTunes library.
4. Patron checks out 20 audio CDs (audio book) and listens to them in a CD player.
5. Patron copies 20 checked out music CDs to a computer at home.
6. Patron starts copying those 20 checked out audio CDs to a laptop in the library.
7. Patron copies the 20 audio CDs at home and then deletes the files after three weeks once the book is due back to the library.
8. Patron copies the 20 audio CDs at home and then deletes the files after six months once the person is done listening to the book.
9. Patron copies the 20 audio CDs at home, listens to the book, and forgets to delete the files from the iPod.
10. Patron purchased a Beatles album in 1964, now copies the same CD to their iPod.
11. Patron purchased a Journey tape in 1983, now copies the same CD to their iPod.
12. Patron purchased a Rush CD in 1996, lost the CD, and copies the same library CD to their iPod.
13. Patron starts copying lots of CDs from the library collection within the library without checking the materials out.

I am sure there are plenty of other scenarios, but I am not sure at which point I should approach the patron.  Even with the egregious case where the person is copying 20 CDs to a laptop within the library, the person could only be intending to load the music or audio book for the next 2-3 weeks for the checkout period.  I have no idea how long the person plans on keeping that information.  [I know that I could ask.] When should I warn the patron about copyright law?  

My reply to the "Public Access to Digital Data" RFI

Here it is.  I wish I had more time to be comprehensive, but this is what I had time to write.  Better this than nothing.

Hello Ted Wackler,

I am writing to the OSTP office concerning the “Request for Information: Public Access to Digital Data Resulting From Federally Funded Scientific Research” that is available at http://www.federalregister.gov/articles/2011/11/04/2011-28621/request-for-information-public-access-to-digital-data-resulting-from-federally-funded-scientific.

I will put in my comments after the numbered sections below. 

Preservation, Discoverability, and Access 

(1) What specific Federal policies would encourage public access to and the preservation of broadly valuable digital data resulting from federally funded scientific research, to grow the U.S. economy and improve the productivity of the American scientific enterprise?

I would like to see PubMed Central (http://www.ncbi.nlm.nih.gov/pmc/) include more data as well as journal articles.  With the new NSF data management plan requirements, research done with NSF funds could copy the data to an NSF repository.  I would also like to see expanded roles for NTIS and the DOE Information Bridge in holding more data from research.  I know that NTIS often sells their reports, but it would be better if the reports and data were freely available to the general public. Astronomical data could be held at the NASA ADS with greater Federal support, http://adsabs.harvard.edu/index.html

(2) What specific steps can be taken to protect the intellectual property interests of publishers, scientists, Federal agencies, and other stakeholders, with respect to any existing or proposed policies for encouraging public access to and preservation of digital data resulting from federally funded scientific research?

Where applicable, I would recommend that Federally funded research license their material with a CC by license (http://creativecommons.org/licenses/by/3.0/) or CC0 (http://creativecommons.org/publicdomain/zero/1.0/).  This will provide the widest reach to readers throughout the whole world.  This will also have the most benefit for scientists, federal agencies, the readers and the citizens of the United States.  It may not be as beneficial for commercial publishers, but they have plenty of other non-government sponsored material they can publish. 

(3) How could Federal agencies take into account inherent differences between scientific disciplines and different types of digital data when developing policies on the management of data?

There are many different data types.  The Global Change Master Directory provides recommendations to scientists who deposit data to the directory.  They provide guides to their metadata writers (Directory Interchange Format (DIF) Writer's Guide). See http://gcmd.nasa.gov/User/difguide/WRITEADIF.pdf and http://gcmd.nasa.gov/User/difguide/difman.html.  This guide could be used as a template to help data management writers describe datasets in other disciplines.

The Digital Curation Centre is another good resource to consult, http://www.dcc.ac.uk/resources/data-management-plans. This is another good resource, “National initiatives for promoting data management strategies: an overview,” http://sonexworkgroup.blogspot.com/2011/04/national-initiatives-for-promoting-data.html

(4) How could agency policies consider differences in the relative costs and benefits of long-term stewardship and dissemination of different types of data resulting from federally funded research?

It depends on who needs to use that data, and the intended audience of the research.

(5) How can stakeholders (e.g., research communities, universities, research institutions, libraries, scientific publishers) best contribute to the implementation of data management plans?

There are many librarians who are getting to be a lot more familiar with data management plans and e-science.  I would recommend that the government work with university programs such as those listed at http://www.arl.org/rtl/eresearch/escien/nsf/nsfresources.shtml.

(6) How could funding mechanisms be improved to better address the real costs of preserving and making digital data accessible?

I am not sure.

(7) What approaches could agencies take to measure, verify, and improve compliance with Federal data stewardship and access policies for scientific research? How can the burden of compliance and verification be minimized?

Scientists need positive reinforcement for depositing and describing their data.  If they received more grant funding for cooperating in projects, or if they received greater recognition by university administrators, then that would be some positive rewards for compliance.

(8) What additional steps could agencies take to stimulate innovative use of publicly accessible research data in new and existing markets and industries to create jobs and grow the economy?

There are always more mashups that could be done with GIS data and social science data.

(9) What mechanisms could be developed to assure that those who produced the data are given appropriate attribution and credit when secondary results are reported?

Data sets could be given a permanent citation link, such as a DOI. http://www.doi.org/ I would recommend that you read some of the papers presented at this conference, http://sites.nationalacademies.org/PGA/brdi/PGA_064019 “Developing Data Attribution and Citation Practices and Standards: An International Symposium and Workshop”

Standards for Interoperability, Re-Use and Re-Purposing

(10) What digital data standards would enable interoperability, reuse, and repurposing of digital scientific data? For example, MIAME (minimum information about a microarray experiment; see Brazma et al., 2001, Nature Genetics 29, 371) is an example of a community-driven data standards effort.

This chapter might be of use to you. http://www.ncbi.nlm.nih.gov/books/NBK45678/ “The Current State of Data Integration in Science” found in the book, Steps Toward Large-Scale Data Integration in the Sciences: Summary of a Workshop. National Research Council (US) Committee on Applied and Theoretical Statistics. http://www.nap.edu/catalog.php?record_id=12916

(11) What are other examples of standards development processes that were successful in producing effective standards and what characteristics of the process made these efforts successful?

I can’t find any right now.

(12) How could Federal agencies promote effective coordination on digital data standards with other nations and international communities?

Start with one country, and then start working with other countries.  I’d recommend that you take a look at the policies of the United Kingdom. Consider looking at http://www.dcc.ac.uk/resources/policy-and-legal/policy-tools-and-guidance and http://www.jiscdigitalmedia.ac.uk/crossmedia/advice/establishing-a-digital-preservation-policy/.

(13) What policies, practices, and standards are needed to support linking between publications and associated data?

I would recommend that you take a look at this article, http://www.plosone.org/article/info:doi/10.1371/journal.pone.0021101 for some practices that are used.