[seqfan] TOPIC: OEIS XML

[Chuck Seggelin]

This all sounds good to me.  XML is far more portable.  I'd like to add only 
one suggestion for consideration:

If the OEIS went to XML format, it would probably be a very small jump to 
give it an RSS feed.  That way subscribers could be automatically notified 
whenever the OEIS DB was updated, and would be able to see exactly what 
updates happened using a news aggregator like bloglines or what have you.

                -- Chuck

----- Original Message ----- 
From: "David Wilson" <davidwwilson at comcast.net>
To: "Antti Karttunen" <Antti.Karttunen at iki.fi>
Cc: "Sequence Fans" <seqfan at ext.jussieu.fr>
Sent: Friday, April 22, 2005 5:46 AM
Subject: [seqfan] Re: TOPIC: OEIS XML


> From now on, I will use TDB to stand for the current text EIS database
> and XDB for the proposed XML EIS database.
>
> I have looked over your tentative requirements from an earlier message.
> Many of your requirements are stated in terms of possible XML
> solutions, I have tried to strip these implementation details or solutions
> and summarize the requirements:
>
> 1.    The XDB should be easier for NJAS to maintain than the TDB.
> 2.    The transition from TDB to XDB should be minimally painful.
> 3.    The XDB should be normalized.
> 4.    The XDB should support something akin to source control,
>    tracking contributors and modification times per datum.
> 5.    The XDB should distinguish between definitions, identities, and
>    conjectures.
> 5a.   The XDB should track contributors and sources per datum.
> 6.     The XDB should support a simple language for defining sequences
>    in terms of other sequences, common sequence classes, and common
>    operations on sequences.
> 7.     The XDB should support equations such as those provided for
>    MathML or TeX.
> 8.    The XDB data organization should improve on that of the TDB.
> 9.    The XDB should support data in formats transitional between the
>    TDB format and the XDB format.
>
> I would categorize these requirements as follows, and add some of my
> own requirements:
>
> I.  Implementation requirements:
>    A.  Migration from TDB to XDB should be as seamless as possible.
>
> I.  Functional requirements:
>    A.  The XDB must support all current functions of the TDB.
>    B.  The XDB should support some form of source control, with
>        automatic tracking of modifications, including submitter,
>        data/time, and source documents associated with the modification.
>    C.  The XDB should support a simple, broad sequence definition
>        language.
>    D.  The XDB should support equations similar to MathML or
>        TeX equations.
>
> II.  Format requirements:
>    A.  The XDB should support data formats that are transitional
>        between the TDB and XDB.
>    B.  The XDB data model should reflect as closely as is practical
>        the perceived organization of the data.
>    C.  The XDB should be normalized.
>
> This requirements list is a prototype.  However, even with this much
> data, I can safely say the that project must be implemented in stages,
> with each stage implementing a chewable number of features.  As a
> software engineer, my approach to this project would be approximately
> as follows:
>
> 1.  Define an API for the OEIS database.  This involves surveying the
>    current functionality of the TDB.  We need to know everything that
>    NJAS, editors, the site, and users do with the database, since we
>    need to maintain all those functions in the TDB.  For instance, once
>    we have transistioned to the TDB, NJAS will still want to create and
>    tweak sequences, the site will still want to look up sequences by the
>    existing criteria, and users will still want to be able to download
>    sections of the database.  Loss of any significant function will
>    conflict with the requirement of seamless migration, and will be
>    disruptive to the operation of the OEIS.
>
> 2.  Once the API is defined, design a transitional XDB whose internal
>    organization is faithful to the current TDB.  I would suggest that
>    TDB-style tags be in their own namespace so that they will not
>    conflict with XDB-style flags later on.
>
> 3.  Implement the API on the transitional XDB containing only
>    TDB-style tags.  This will prove that the XDB is complete datawise
>    and provide a test bed for future XDB development.
>
> 4.  Design the XDB-style tags and elements.  Survey the TDB data and
>    proposed enhancements.  XDB-style tags and elements should be in
>    their own name space.
>
> 5.  Implement the XDB-style tags.  As each tag or group of related
>    XDB-style tags is implemented, the API should be modified to deal
>    with the XDB-style tags.  It may be expedient at this point to convert
>    TDB-style tags to XDB-style tags, or else teach the API to deal with
>    mixed tags and postpone the conversion.  When the API is fully
>    functional, we have effectively replaced the TDB, and now have
>    a database with XML functionality, to which more-or-less seamless
>    enhancements can be made.
>
> 6.  Create source control tools for the TDB.  These tools should
>   allow users to create new sequences or check out existing sequences
>   for modification, submit them for review, assign them to a reviewer,
>   guide the reviewer through the review process and approve the
>   sequence, and accept the sequence into the database (or at any
>   point reject the sequence, which will then not enter the database).
>   When the sequence is accepted into the database, the contributor
>   and date/time are added to the modified lines.
>
> 7.  Think about enhancements, such as MathML equations or
>   simple sequence descriptions.
>
> ----- Original Message ----- 
> From: "Antti Karttunen" <Antti.Karttunen at iki.fi>
> To: <ham>; "David Wilson" <davidwwilson at comcast.net>; "Hugo van der 
> Sanden" <hv at crypt.org>; "Gerald McGarvey" <Gerald.McGarvey at comcast.net>; 
> "Ralf Stephan" <ralf at ark.in-berlin.de>; "Thomas Baruchel" 
> <baruchel at laposte.net>; "Marc LeBrun" <mlb at fxpt.com>; "Antti Karttunen" 
> <Antti.Karttunen at iki.fi>
> Sent: Thursday, April 21, 2005 4:21 PM
> Subject: Re: TOPIC: OEIS XML
>
>
>> David Wilson wrote:
>>
>>> To begin, I think the conversion of the OEIS into XML is a great
>>> idea.  I would, however, like to understand the goal of the project.
>>> Are we merely trying to emulate the current sequence schema in
>>> XML, or are we going to develop a new schema for the XML
>>> database, that is, the way the sequences SHOULD be structured?
>>>
>>> If the former, the whole project seems pretty cut and dried, and
>>> you should need no participation beyond the techical people.  If
>>> the latter, I suggest that you start putting together requirements
>>> and/or design documents that can be reviewed and criticized by
>>> us XML illiterates.
>>
>>
>> Here are some of my thoughts.
>>
>> First of all, I'm here thinking about the full-scale replacement
>> of the current implementation of the OEIS-system, based
>> on the data stored in RDBMS or other database which
>> can give an XML-view to its data. This would involve
>> also all the operations Neil has to do for the maintenance
>> of OEIS, currently invisible to us.
>>
>> [list of requirements omitted]
>
>