gmod-schema Mailing List for Generic Model Organism Database Project

On Wednesday, July 30, 2003, at 10:29 AM, Allen Day wrote:
> It shouldn't be too hard to autogenerate the simple bridging where 
> there
> is a one-to-one correspondence between chado tables (or chado star
> subschemas) and bioperl classes.
BTW this is exactly what bioperl-db does to a large extent where there 
is such a straightforward correspondence. It only does it at runtime, 
it doesn't generate classes you'd have to maintain.
	-hilmar
BTW in reality it turns out that there are really only very few classes 
that map directly to tables without special case code. It's not a new 
finding that relational and object models are different animals. People 
tend to forget that. I'd be curious to learn how you can auto-generate 
an at least minimally useful OR mapper from bioperl to any of chado, 
biosql, or you name it.
Literally all of the generic OR mappers that I'm aware of either a) 
assume you've got the schema and generate the object model and the 
mapping off of it, or b) assume you've got the object model and 
generate the schema and the mapping off of it, or c) require an 
extensive amount of hard to debug (which is a euphemism here) 
configuration files.
Here at GNF we've been going through exercise c) with J2EE CMP as 
implemented by Jboss. We're not going to do it again. Auto-generating 
the OR mapping either hand-cuffs one arm onto your back, or it's just 
not worth the nightmare of staring at seemingly correct configuration 
files.
-- 
-------------------------------------------------------------
Hilmar Lapp email: lapp at gnf.org
GNF, San Diego, Ca. 92121 phone: +1-858-812-1757
-------------------------------------------------------------

Hi Nat,
On Wednesday, July 30, 2003, at 05:40 AM, Nathan ((Nat)) Goodman wrote:
> Thanks, everyone, for the helpful comments!
>
> Hilmar wrote:
>> I gave an extensive talk at BOSC03 on biosql and bioperl-db, and I can
>> email you the slides if you are interested.
>
> Yes, the slides would be great!
I'll send them along, and also to everyone else who emailed me to 
request them. Or somebody can refresh my memory on how I put something 
up on the website for download; that'd probably be the easiest.
> I'm starting to feel my way through the
> code. I gather that to make class XXX persistent, I create a
> Bio::DB::BioSQL::XXXAdapter class, yes? Seems simple enough... 
> (famous last
> words :)
Yes, if you're talking about making a class persistent that isn't yet. 
There is an abstract base class Bio::DB::BioSQL::BasePeristenceAdaptor 
that you inherit from and that implements all the mundane stuff. You 
just have to override those methods that the base implementation can't 
figure out for you, like what are the object properties to be 
persisted, what are the objects it depends on in a foreign key sense, 
and how to get and how to set the properties' values.
If you're talking about making an object persistent at runtime 
(provided the class is supported by the adaptors), it's simpler than 
that:
	$db = Bio::DB::BioDB->new(-database => "biosql", /* connection params 
*/);
	$my_persistent_obj = $db->create_persistent($my_obj);
	# then insert
	$my_persistent_obj->create();
	# or manipulate then update
	$my_persistent_obj->store();
	# or find an existing one
	$adp = $db->get_object_adaptor($my_obj);
	$my_persistent_object = $adp->find_by_unique_key($my_obj);
	# either manipulate and update, or e.g. delete
	$my_persistent_object->remove();
>
> I should have made my needs a bit more clear. I want to use existing
> schemas and adapters for existing BioPerl classes. I accept the
> responsibility to create schemas and adapters for new BioPerl (or 
> BioPerl
> wannabe) classes I write, and hope that others will do the same for 
> BioPerl
> classes they write.
Bioperl-db doesn't support all bioperl objects yet, only the sequence 
and annotation related ones: sequences, clusters, features, locations, 
all of the annotation bundle, ontologies, species.
One of the things in bioperl-db is now that you have a nice framework 
for setting up your own persistence adaptor, requiring you to only pay 
attention to those things that are special about the class you want to 
persist.
I should probably add a disclaimer that that doesn't mean that 
bioperl-db is simple code that can be easily understood in its 
entirety. It's a relatively generic OR mapper, and learning to 
understand the flow of instruction is maybe a daunting task at times. 
On the other hand, the base implementation provides lots of trace 
output if verbose mode is enabled, so it's usually not hard to get a 
good picture of what's going on if you need to.
> I expect to access the database through BioPerl only
> (except for occasional admin purposes) which means that the only 
> features
> I'll ever use are ones that are visitble through BioPerl.
>
Then maybe the combination of biosql and bioperl-db is for you.
> I'm trying to hitch my wagon to the schema that has the most mature 
> BioPerl
> binding and that is likely to be most widely adopted by the community 
> of
> BioPerl developers (so that the BioPerl binding will continue to 
> improve).
I'm not sure about the current level of adoption of biosql among 
bioperl developers. It used to be relatively quiet in summer last year, 
which was when I picked it up. Given the list activity more recently, 
it seems to have picked up interest at a number of places. Among people 
closer to the bioperl core, except for myself I know that Elia's group 
has been using it, and Ewan's used it to replace SRS.
Ewan also spent an extensive amount of time at the last hackathon to 
try and document how to set up a working biosql/bioperl-db environment 
from scratch, including installing mysql or Pg (which took most of the 
time I believe) and instantiating the schema.
	-hilmar
-- 
-------------------------------------------------------------
Hilmar Lapp email: lapp at gnf.org
GNF, San Diego, Ca. 92121 phone: +1-858-812-1757
-------------------------------------------------------------

> > There is a perl O/R layer for chado -- it's autogenerated Class::DBI code
> > from the CREATE TABLE statements. There are not yet any adapters written
> > that convert the Class::DBI objects to/from bioperl objects.
> 
> Yes of course, sorry, totally spaced on this part, I've been focused on
> the components developed by the harvard/flybase folks.
> 
> Is there any plans to write the bioperl bridge anytime soon?
Not in the immediate future, as I don't really have an urgent need to
bridge at this time. If I had time, I'd start with the cvterm/ontology
and rad(expression)/bioperl-microarray bridges.
It shouldn't be too hard to autogenerate the simple bridging where there
is a one-to-one correspondence between chado tables (or chado star
subschemas) and bioperl classes. We just need to write a configuration
file that maps chado tables/columns to bioperl classes/attributes and hand
it, along with either a chado or bioperl object, to a generic factory
class that knows how to do the interconversion.
The more complex relations in chado (eg features graphs) are going to be
harder to bridge because chado and bioperl model these relations
differently.
-Allen

On 2003年7月29日, Allen Day wrote:
> > BioSQL certainly has the tightest integration with bioperl (and the other
> > bio* projects). This is through an O/R layer.
> >
> > chado has no direct integration with bioperl. I don't think there is any
> > O/R layer or OO API planned (although some biojava folks have expressed an
> > interest in this), Scott Cain has written a chado adapter for gbrowse
> > (which uses bioperl objects) which could be extracted to form an API in
> > its own right (although it is currently limited to the kind of API
> > calls you need to make a genome viewer).
> >
> > Many of the chado developers favour XML over objects. Chado-XML DTD is
> > derived directly from the relational schema. The chado developers at
> > Harvard have written a generic XML<->DB tool, which can be used in place
> > of an API or O/R mapping. Of course, we still want to be able to use
> > bioperl objects, so there are Bio::SeqIO::chadoxml classes being
> > developed. The most likely route will be DB<->ChadoXML<->bioperl.
>
> There is a perl O/R layer for chado -- it's autogenerated Class::DBI code
> from the CREATE TABLE statements. There are not yet any adapters written
> that convert the Class::DBI objects to/from bioperl objects.
Yes of course, sorry, totally spaced on this part, I've been focused on
the components developed by the harvard/flybase folks.
Is there any plans to write the bioperl bridge anytime soon?
> -Allen
>
>

In a message dated 7/29/2003 11:54:06 PM Eastern Standard Time, 
all...@uc... writes:
> >Many of the chado developers favour XML over objects. Chado-XML DTD is
> >derived directly from the relational schema. The chado developers at
> >Harvard have written a generic XML<->DB tool, which can be used in place
> >of an API or O/R mapping. Of course, we still want to be able to use
> >bioperl objects, so there are Bio::SeqIO::chadoxml classes being
> >developed. The most likely route will be DB<->ChadoXML<->bioperl.
> 
> There is a perl O/R layer for chado -- it's autogenerated Class::DBI code
> from the CREATE TABLE statements. There are not yet any adapters written
> that convert the Class::DBI objects to/from bioperl objects.
> 
> 
Peili Zhang at FlyBase Harvard, who is currently vacationing in the French 
countryside,
has developed a BioPerl<->chadoXML module. It is not 100% complete -- there
were some issues about how to map to less expressive formalisms such
as GenBank. If anyone is interested in details I suggest they contact her in 
2 weeks.
Cheers, -Stan

Hi Hilmar
As Chris mentioned I'm investigating how feasible it would be to bind 
Bioperl and GUS using the Bioperl-db interfaces implementation.
Could you sent me the slides from your talk at the BOSC as well ? I 
think they will be handy.
Thanks
Arnaud
Chris Stoeckert wrote:
> Hi Nat,
> Coming into this exchange late but just wanted to mention our plans 
> for BioPerl and GUS. GUS has its own object layer but we are 
> exploring BioPerl for transferring standard sequence analyses into 
> the GUS tables. Arnaud Kerhornou (GeneDB) at the Sanger Institute is 
> working on this.
>
> Cheers,
> Chris
>
> On Wednesday, July 30, 2003, at 08:40 AM, Nathan ((Nat)) Goodman wrote:
>
>> Thanks, everyone, for the helpful comments!
>>
>> Hilmar wrote:
>>
>>> I gave an extensive talk at BOSC03 on biosql and bioperl-db, and I can
>>> email you the slides if you are interested.
>>
>>
>> Yes, the slides would be great! I'm starting to feel my way through 
>> the
>> code. I gather that to make class XXX persistent, I create a
>> Bio::DB::BioSQL::XXXAdapter class, yes? Seems simple enough... 
>> (famous last
>> words :)
>>
>> I should have made my needs a bit more clear. I want to use existing
>> schemas and adapters for existing BioPerl classes. I accept the
>> responsibility to create schemas and adapters for new BioPerl (or 
>> BioPerl
>> wannabe) classes I write, and hope that others will do the same for 
>> BioPerl
>> classes they write. I expect to access the database through BioPerl 
>> only
>> (except for occasional admin purposes) which means that the only 
>> features
>> I'll ever use are ones that are visitble through BioPerl.
>>
>> I'm trying to hitch my wagon to the schema that has the most mature 
>> BioPerl
>> binding and that is likely to be most widely adopted by the 
>> community of
>> BioPerl developers (so that the BioPerl binding will continue to 
>> improve).
>>
>> Thanks,
>> Nat
>>

Hi Nat,
Coming into this exchange late but just wanted to mention our plans for 
BioPerl and GUS. GUS has its own object layer but we are exploring 
BioPerl for transferring standard sequence analyses into the GUS 
tables. Arnaud Kerhornou (GeneDB) at the Sanger Institute is working on 
this.
Cheers,
Chris
On Wednesday, July 30, 2003, at 08:40 AM, Nathan ((Nat)) Goodman wrote:
> Thanks, everyone, for the helpful comments!
>
> Hilmar wrote:
>> I gave an extensive talk at BOSC03 on biosql and bioperl-db, and I can
>> email you the slides if you are interested.
>
> Yes, the slides would be great! I'm starting to feel my way through 
> the
> code. I gather that to make class XXX persistent, I create a
> Bio::DB::BioSQL::XXXAdapter class, yes? Seems simple enough... 
> (famous last
> words :)
>
> I should have made my needs a bit more clear. I want to use existing
> schemas and adapters for existing BioPerl classes. I accept the
> responsibility to create schemas and adapters for new BioPerl (or 
> BioPerl
> wannabe) classes I write, and hope that others will do the same for 
> BioPerl
> classes they write. I expect to access the database through BioPerl 
> only
> (except for occasional admin purposes) which means that the only 
> features
> I'll ever use are ones that are visitble through BioPerl.
>
> I'm trying to hitch my wagon to the schema that has the most mature 
> BioPerl
> binding and that is likely to be most widely adopted by the community 
> of
> BioPerl developers (so that the BioPerl binding will continue to 
> improve).
>
> Thanks,
> Nat
>
>
>
> -------------------------------------------------------
> This SF.Net email sponsored by: Free pre-built ASP.NET sites including
> Data Reports, E-commerce, Portals, and Forums are available now.
> Download today and enter to win an XBOX or Visual Studio .NET.
> http://aspnet.click-url.com/go/psa00100003ave/ 
> direct;at.aspnet_072303_01/01
> _______________________________________________
> Gmod-schema mailing list
> Gmo...@li...
> https://lists.sourceforge.net/lists/listinfo/gmod-schema
>

I'm coming in on the cross-post, so please forgive me if I'm repeating
something thats already been said...
On 07/30/03 at 12:47 AM Ewan Birney wrote:
>
> >
> > ensembl is a different kettle of fish altogether. The main
difference is
> > that typing is enforced at the relational layer in ensembl. this
has many
> > advantages and disadvantages which have been discussed to death, it
> > depends on your project really.
> >
> > ensembl is the most mature, and chado is the new kid on the block.
> > however, chado 1_01 has just been frozen, and that's what most apps
will
> > be targetting.
> >
>
> ensembl is also strictly targetted at "storing features on genome
> sequence" whereas BioSQL has a broader "storing biological stuff"
target.
> Ensembl has been in fact drifting away from Bioperl towards just
> supporting an extremely clean Perl and Java API on-top of its
> highly-relational focused schema. The bioperl compatibility is now
really
> around the edges. I may, over the same, build a explicit standalone
bridge
> between Ensembl and Bioperl which would provide all the usual Bioperl
> interfaces from this pretty vanilla schema.
>
>
> So...
>
> Storing Just Genomes --- choose between Chado and Ensembl, with
> undoubtly Ensembl having the current most mature tool sets (including
> loaders now from particular flavours of GenBank/EMBL format)
>
> Storing "any" sequence --- BioSQL or roll your own.
>
Exactly right. And what distinguishes chado from ensembl is that chado
is being designed to integrate genetic/phenotypic literature curation
data with genome data.
With respect to genome sequence and annotation data in chado, bioperl
bindings are only now emerging. The genetic module of chado is itself
still evolving, and as far as bioperl goes, being extremely sequence/
annotation-centric, there are no classes to handle this sort of data.
Once we've got the chado genetic module implemented, it'll be very
interesting to see how bioperl might be extended to handle integrated
genomic/genetic data.
-Dave
>
>
>
> -------------------------------------------------------
> This SF.Net email sponsored by: Free pre-built ASP.NET sites
including
> Data Reports, E-commerce, Portals, and Forums are available now.
> Download today and enter to win an XBOX or Visual Studio .NET.
> http://aspnet.click-url.com/go/psa00100003ave/
direct;at.aspnet_072303_01/01
> _______________________________________________
> Gmod-schema mailing list
> Gmo...@li...
> https://lists.sourceforge.net/lists/listinfo/gmod-schema

Thanks, everyone, for the helpful comments!
Hilmar wrote:
> I gave an extensive talk at BOSC03 on biosql and bioperl-db, and I can
> email you the slides if you are interested.
Yes, the slides would be great! I'm starting to feel my way through the
code. I gather that to make class XXX persistent, I create a
Bio::DB::BioSQL::XXXAdapter class, yes? Seems simple enough... (famous last
words :)
I should have made my needs a bit more clear. I want to use existing
schemas and adapters for existing BioPerl classes. I accept the
responsibility to create schemas and adapters for new BioPerl (or BioPerl
wannabe) classes I write, and hope that others will do the same for BioPerl
classes they write. I expect to access the database through BioPerl only
(except for occasional admin purposes) which means that the only features
I'll ever use are ones that are visitble through BioPerl.
I'm trying to hitch my wagon to the schema that has the most mature BioPerl
binding and that is likely to be most widely adopted by the community of
BioPerl developers (so that the BioPerl binding will continue to improve).
Thanks,
Nat

>
> ensembl is a different kettle of fish altogether. The main difference is
> that typing is enforced at the relational layer in ensembl. this has many
> advantages and disadvantages which have been discussed to death, it
> depends on your project really.
>
> ensembl is the most mature, and chado is the new kid on the block.
> however, chado 1_01 has just been frozen, and that's what most apps will
> be targetting.
>
ensembl is also strictly targetted at "storing features on genome
sequence" whereas BioSQL has a broader "storing biological stuff" target.
Ensembl has been in fact drifting away from Bioperl towards just
supporting an extremely clean Perl and Java API on-top of its
highly-relational focused schema. The bioperl compatibility is now really
around the edges. I may, over the same, build a explicit standalone bridge
between Ensembl and Bioperl which would provide all the usual Bioperl
interfaces from this pretty vanilla schema.
So...
 Storing Just Genomes --- choose between Chado and Ensembl, with
undoubtly Ensembl having the current most mature tool sets (including
loaders now from particular flavours of GenBank/EMBL format)
 Storing "any" sequence --- BioSQL or roll your own.

Excellent reply Chris.
There's almost nothing I can meaningfully add. Biosql has essentially 
been frozen since a few months and the only changes during this time 
have been minor bug fixes (actually some bigger ones in the Oracle 
part).
It really depends a lot on your use case(s) and what your pool of 
expertise is on which you can rely.
As for the tight language bindings of biosql in bioperl, the OR layer 
is relatively flexible (with some limitations) in what the underlying 
schema is. If you guys should choose chado but still want to have the 
tight language binding, I can guide as to how best to extend bioperl-db 
to persist to chado; this has been on my wish list for a while.
I guess one of the advantages of biosql is the bio* support (there are 
bindings in biojava, biophython, and bioruby, apart from bioperl), but 
the question is whether this helps you. Also, biosql is supported on 
mysql, Pg, and Oracle, but again this may or may not help you.
I'm also x-posting this to biosql-l, in case someone wants to report 
experiences with using biosql or share testimonials ...
I gave an extensive talk at BOSC03 on biosql and bioperl-db, and I can 
email you the slides if you are interested. (Chris, is there actually a 
website for speakers to post their slides?)
	-hilmar
On Tuesday, July 29, 2003, at 06:31 PM, Chris Mungall wrote:
>
> [x-posting to GMOD-schema]
>
> On 2003年7月29日, Nathan (Nat) Goodman wrote:
>
>> I'm thinking about converting our homegrown relational schema to one 
>> of the
>> emerging BioPerl-friendly "standard" schemas. I'm looking for 
>> something
>> that (1) works now, and (2) is likely to be popular in the BioPerl 
>> world for
>> some time to come.
>>
>> I think the choices are BioSQL and chado. Are there others? Is one 
>> of
>> these the obvious right choice?
>
> ensembl and GUS are the other main choices
>
> I originally viewed BioSQL as a way of doing relational queries over 
> data
> slurped from EMBL/GenBank and SwissProt. It has since evolved into
> something more generic and resembles chado in many ways. Hilmar and 
> Dave
> Block use it at GNF all the time for a lot more than slurping genbank.
>
> Chado encompasses more than BioSQL - genetics, expression, 
> publications.
> But I'm assuming it's the sequence part you are interested in here? 
> There
> is nothing to stop BioSQL moving into this area.
>
> BioSQL certainly has the tightest integration with bioperl (and the 
> other
> bio* projects). This is through an O/R layer.
>
> chado has no direct integration with bioperl. I don't think there is 
> any
> O/R layer or OO API planned (although some biojava folks have 
> expressed an
> interest in this), Scott Cain has written a chado adapter for gbrowse
> (which uses bioperl objects) which could be extracted to form an API in
> its own right (although it is currently limited to the kind of API
> calls you need to make a genome viewer).
>
> Many of the chado developers favour XML over objects. Chado-XML DTD is
> derived directly from the relational schema. The chado developers at
> Harvard have written a generic XML<->DB tool, which can be used in 
> place
> of an API or O/R mapping. Of course, we still want to be able to use
> bioperl objects, so there are Bio::SeqIO::chadoxml classes being
> developed. The most likely route will be DB<->ChadoXML<->bioperl.
>
> BioSQL is semantically almost identical to the bioperl object model,
> whereas there are some differences with chado, specifically with 
> respect
> to locations.
>
> chado does not allow discontinuous/split feature locations
> chado does not support the full fuzzy genbank model
> chado allows multiple redundant locations
> (eg a SNP on a protein vs genomic; features on clone and chromosome)
> chado uses interbase
> chado uses a different mechanism for 'remote' locations
> the source feature (ie the one which start/end is relative to)
> is part of the location in chado, unlike bioSQL
> chado abandons the artifical distinction between 'sequence' and 
> 'feature',
> there is only one entity 'feature' in the _logical_ model
> chado has no equivalent of biosql.bioentry (other than 'feature')
>
> These aspects of chado are more fully documented in the sql ddl, and 
> in a
> document which is.... Stan/Dave.... where abouts is that doc?
>
> other than that, there are more similarities than differences. eg Both
> allow arbitrary feature graphs (preferably conforming to SO 
> partonomies),
> features are typed by an ontology etc. I'm sure migrating data one way 
> or
> the other wouldn't be too much of a problem.
>
> ensembl is a different kettle of fish altogether. The main difference 
> is
> that typing is enforced at the relational layer in ensembl. this has 
> many
> advantages and disadvantages which have been discussed to death, it
> depends on your project really.
>
> ensembl is the most mature, and chado is the new kid on the block.
> however, chado 1_01 has just been frozen, and that's what most apps 
> will
> be targetting.
>
> chado has a lot riding on it right now; flybase will become completely
> chado-dependent for its genome annotation data by the end of this year.
>
>> Thanks,
>> Nat
>
> Cheers
> Chris
>
> _______________________________________________
> Bioperl-l mailing list
> Bio...@po...
> http://portal.open-bio.org/mailman/listinfo/bioperl-l
>
-- 
-------------------------------------------------------------
Hilmar Lapp email: lapp at gnf.org
GNF, San Diego, Ca. 92121 phone: +1-858-812-1757
-------------------------------------------------------------

> BioSQL certainly has the tightest integration with bioperl (and the other
> bio* projects). This is through an O/R layer.
> 
> chado has no direct integration with bioperl. I don't think there is any
> O/R layer or OO API planned (although some biojava folks have expressed an
> interest in this), Scott Cain has written a chado adapter for gbrowse
> (which uses bioperl objects) which could be extracted to form an API in
> its own right (although it is currently limited to the kind of API
> calls you need to make a genome viewer).
> 
> Many of the chado developers favour XML over objects. Chado-XML DTD is
> derived directly from the relational schema. The chado developers at
> Harvard have written a generic XML<->DB tool, which can be used in place
> of an API or O/R mapping. Of course, we still want to be able to use
> bioperl objects, so there are Bio::SeqIO::chadoxml classes being
> developed. The most likely route will be DB<->ChadoXML<->bioperl.
There is a perl O/R layer for chado -- it's autogenerated Class::DBI code
from the CREATE TABLE statements. There are not yet any adapters written
that convert the Class::DBI objects to/from bioperl objects.
-Allen

[x-posting to GMOD-schema]
On 2003年7月29日, Nathan (Nat) Goodman wrote:
> I'm thinking about converting our homegrown relational schema to one of the
> emerging BioPerl-friendly "standard" schemas. I'm looking for something
> that (1) works now, and (2) is likely to be popular in the BioPerl world for
> some time to come.
>
> I think the choices are BioSQL and chado. Are there others? Is one of
> these the obvious right choice?
ensembl and GUS are the other main choices
I originally viewed BioSQL as a way of doing relational queries over data
slurped from EMBL/GenBank and SwissProt. It has since evolved into
something more generic and resembles chado in many ways. Hilmar and Dave
Block use it at GNF all the time for a lot more than slurping genbank.
Chado encompasses more than BioSQL - genetics, expression, publications.
But I'm assuming it's the sequence part you are interested in here? There
is nothing to stop BioSQL moving into this area.
BioSQL certainly has the tightest integration with bioperl (and the other
bio* projects). This is through an O/R layer.
chado has no direct integration with bioperl. I don't think there is any
O/R layer or OO API planned (although some biojava folks have expressed an
interest in this), Scott Cain has written a chado adapter for gbrowse
(which uses bioperl objects) which could be extracted to form an API in
its own right (although it is currently limited to the kind of API
calls you need to make a genome viewer).
Many of the chado developers favour XML over objects. Chado-XML DTD is
derived directly from the relational schema. The chado developers at
Harvard have written a generic XML<->DB tool, which can be used in place
of an API or O/R mapping. Of course, we still want to be able to use
bioperl objects, so there are Bio::SeqIO::chadoxml classes being
developed. The most likely route will be DB<->ChadoXML<->bioperl.
BioSQL is semantically almost identical to the bioperl object model,
whereas there are some differences with chado, specifically with respect
to locations.
chado does not allow discontinuous/split feature locations
chado does not support the full fuzzy genbank model
chado allows multiple redundant locations
 (eg a SNP on a protein vs genomic; features on clone and chromosome)
chado uses interbase
chado uses a different mechanism for 'remote' locations
 the source feature (ie the one which start/end is relative to)
 is part of the location in chado, unlike bioSQL
chado abandons the artifical distinction between 'sequence' and 'feature',
 there is only one entity 'feature' in the _logical_ model
chado has no equivalent of biosql.bioentry (other than 'feature')
These aspects of chado are more fully documented in the sql ddl, and in a
document which is.... Stan/Dave.... where abouts is that doc?
other than that, there are more similarities than differences. eg Both
allow arbitrary feature graphs (preferably conforming to SO partonomies),
features are typed by an ontology etc. I'm sure migrating data one way or
the other wouldn't be too much of a problem.
ensembl is a different kettle of fish altogether. The main difference is
that typing is enforced at the relational layer in ensembl. this has many
advantages and disadvantages which have been discussed to death, it
depends on your project really.
ensembl is the most mature, and chado is the new kid on the block.
however, chado 1_01 has just been frozen, and that's what most apps will
be targetting.
chado has a lot riding on it right now; flybase will become completely
chado-dependent for its genome annotation data by the end of this year.
> Thanks,
> Nat
Cheers
Chris

Stan, 
I understand and sympathize. I wish there was something I could do to
help, but I doubt that there is :-/
Scott
On Mon, 2003年07月28日 at 16:59, SLe...@ao... wrote:
> In a message dated 7/28/2003 2:08:29 PM Eastern Standard Time,
> ca...@cs... writes:
> 
> > For some reason, I thought the
> > Apollo-chado connection was farther along than that. My goal really
> > is
> > to have a release of GMOD ready by the meeting in September, but I
> > must
> > say the time line you give makes me a little nervous.
> 
> 
> Scott,
> We had the components largely done a while ago, but integration
> testing
> has been very time consuming. The timeline is not because we are doing
> other
> things but because it really is a bear to get right.
> 
> Cheers, -Stan
-- 
------------------------------------------------------------------------
Scott Cain, Ph. D. ca...@cs...
GMOD Coordinator (http://www.gmod.org/) 216-392-3087
Cold Spring Harbor Laboratory

Allen,
That is good news. I figured not much has been happening since there
has been little cvs activity on gmod-web. Let me know if there is
anything I can do to help. (For the first time last weekend, I compiled
my own Apache/mod_perl.)
Scott
On Mon, 2003年07月28日 at 14:21, Allen Day wrote:
> all,
> 
> we're in the process of overhauling the web code base right now, trimming
> out as much as possible of the apache 1.3 features that haven't yet been
> ported to 2.0. we're also adding some base classes to make the webserver
> debugging easier.
> 
> because of some advances in sql::translator, we're currently
> autogenerating middleware, and starting to autogenerate read-only web
> templates and we configuration files directly from the chado sql files. 
> query interface templates currently need to be hand coded -- we're still
> waiting to see what Ken Clark comes up with WRT generating web forms from
> CREATE TABLE statements.
> 
> it's conceivable that we will have an alpha release ready by september.
> 
> -allen
> 
> 
> 
> On 28 Jul 2003, Scott Cain wrote:
> 
> > On Mon, 2003年07月28日 at 13:51, David Emmert wrote:
> > > I thought the parts of the first GMOD release were chado, gbrowse, and 
> > > the web interface...?
> > > 
> > > Frank and Pinglei are coding the Apollo->chado cycle right now. Pinglei 
> > > suggested we'd probably be looking at early- to mid- September before we 
> > > could make a beta release available to GMOD.
> > > 
> > > Has the web interface been postponed?
> > > 
> > Dave,
> > 
> > Obviously, having an editor is pretty import for a MOD, so I was hoping
> > to get Apollo in with a first release. For some reason, I thought the
> > Apollo-chado connection was farther along than that. My goal really is
> > to have a release of GMOD ready by the meeting in September, but I must
> > say the time line you give makes me a little nervous.
> > 
> > As for the web front end, I consider it slightly (only slightly!) less
> > important, since gbrowse can double as a rudimentary web front end now
> > that it is its gbrowse_details page. I wanted to keep the first release
> > as simple as possible, since that is already pretty complicated.
> > 
> > Scott
> > 
> > > -Dave
> > > 
> > > On 2003年7月28日 1:16PM -0500, Scott Cain wrote:
> > > > Hello,
> > > >
> > > > I would like to start assembling the first release candidate for GMOD.
> > > > This release would consist of chado, gbrowse, apollo and any necessary
> > > > scripts and other components to make it work well together. Among the
> > > > list of "things to make it work" are:
> > > >
> > > > - a script to load chado from genbank or GFF3
> > > > - scripts to allow Apollo to do the GAME-Chadx-chado tango
> > > > - documentation
> > > > - other things I no doubt haven't thought of yet
> > > >
> > > > I am working on the genbank/GFF3 loader. My question is, what is 
> > > > needed
> > > > to make Apollo do its thing? Is it (reasonably) well documented yet?
> > > >
> > > > Thanks,
> > > > Scott
> > > >
> > > > --
> > > > ------------------------------------------------------------------------
> > > > Scott Cain, Ph. D. 
> > > > ca...@cs...
> > > > GMOD Coordinator (http://www.gmod.org/) 
> > > > 216-392-3087
> > > > Cold Spring Harbor Laboratory
> > > >
> > > >
> > > >
> > > > -------------------------------------------------------
> > > > This SF.Net email sponsored by: Free pre-built ASP.NET sites including
> > > > Data Reports, E-commerce, Portals, and Forums are available now.
> > > > Download today and enter to win an XBOX or Visual Studio .NET.
> > > > http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01
> > > > _______________________________________________
> > > > Gmod-schema mailing list
> > > > Gmo...@li...
> > > > https://lists.sourceforge.net/lists/listinfo/gmod-schema
> > 
-- 
------------------------------------------------------------------------
Scott Cain, Ph. D. ca...@cs...
GMOD Coordinator (http://www.gmod.org/) 216-392-3087
Cold Spring Harbor Laboratory

In a message dated 7/28/2003 2:08:29 PM Eastern Standard Time, ca...@cs... 
writes:
> For some reason, I thought the
> Apollo-chado connection was farther along than that. My goal really is
> to have a release of GMOD ready by the meeting in September, but I must
> say the time line you give makes me a little nervous.
Scott,
 We had the components largely done a while ago, but integration 
testing
has been very time consuming. The timeline is not because we are doing other
things but because it really is a bear to get right.
Cheers, -Stan

all,
we're in the process of overhauling the web code base right now, trimming
out as much as possible of the apache 1.3 features that haven't yet been
ported to 2.0. we're also adding some base classes to make the webserver
debugging easier.
because of some advances in sql::translator, we're currently
autogenerating middleware, and starting to autogenerate read-only web
templates and we configuration files directly from the chado sql files. 
query interface templates currently need to be hand coded -- we're still
waiting to see what Ken Clark comes up with WRT generating web forms from
CREATE TABLE statements.
it's conceivable that we will have an alpha release ready by september.
-allen
On 28 Jul 2003, Scott Cain wrote:
> On Mon, 2003年07月28日 at 13:51, David Emmert wrote:
> > I thought the parts of the first GMOD release were chado, gbrowse, and 
> > the web interface...?
> > 
> > Frank and Pinglei are coding the Apollo->chado cycle right now. Pinglei 
> > suggested we'd probably be looking at early- to mid- September before we 
> > could make a beta release available to GMOD.
> > 
> > Has the web interface been postponed?
> > 
> Dave,
> 
> Obviously, having an editor is pretty import for a MOD, so I was hoping
> to get Apollo in with a first release. For some reason, I thought the
> Apollo-chado connection was farther along than that. My goal really is
> to have a release of GMOD ready by the meeting in September, but I must
> say the time line you give makes me a little nervous.
> 
> As for the web front end, I consider it slightly (only slightly!) less
> important, since gbrowse can double as a rudimentary web front end now
> that it is its gbrowse_details page. I wanted to keep the first release
> as simple as possible, since that is already pretty complicated.
> 
> Scott
> 
> > -Dave
> > 
> > On 2003年7月28日 1:16PM -0500, Scott Cain wrote:
> > > Hello,
> > >
> > > I would like to start assembling the first release candidate for GMOD.
> > > This release would consist of chado, gbrowse, apollo and any necessary
> > > scripts and other components to make it work well together. Among the
> > > list of "things to make it work" are:
> > >
> > > - a script to load chado from genbank or GFF3
> > > - scripts to allow Apollo to do the GAME-Chadx-chado tango
> > > - documentation
> > > - other things I no doubt haven't thought of yet
> > >
> > > I am working on the genbank/GFF3 loader. My question is, what is 
> > > needed
> > > to make Apollo do its thing? Is it (reasonably) well documented yet?
> > >
> > > Thanks,
> > > Scott
> > >
> > > --
> > > ------------------------------------------------------------------------
> > > Scott Cain, Ph. D. 
> > > ca...@cs...
> > > GMOD Coordinator (http://www.gmod.org/) 
> > > 216-392-3087
> > > Cold Spring Harbor Laboratory
> > >
> > >
> > >
> > > -------------------------------------------------------
> > > This SF.Net email sponsored by: Free pre-built ASP.NET sites including
> > > Data Reports, E-commerce, Portals, and Forums are available now.
> > > Download today and enter to win an XBOX or Visual Studio .NET.
> > > http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01
> > > _______________________________________________
> > > Gmod-schema mailing list
> > > Gmo...@li...
> > > https://lists.sourceforge.net/lists/listinfo/gmod-schema
> 

On Mon, 2003年07月28日 at 13:51, David Emmert wrote:
> I thought the parts of the first GMOD release were chado, gbrowse, and 
> the web interface...?
> 
> Frank and Pinglei are coding the Apollo->chado cycle right now. Pinglei 
> suggested we'd probably be looking at early- to mid- September before we 
> could make a beta release available to GMOD.
> 
> Has the web interface been postponed?
> 
Dave,
Obviously, having an editor is pretty import for a MOD, so I was hoping
to get Apollo in with a first release. For some reason, I thought the
Apollo-chado connection was farther along than that. My goal really is
to have a release of GMOD ready by the meeting in September, but I must
say the time line you give makes me a little nervous.
As for the web front end, I consider it slightly (only slightly!) less
important, since gbrowse can double as a rudimentary web front end now
that it is its gbrowse_details page. I wanted to keep the first release
as simple as possible, since that is already pretty complicated.
Scott
> -Dave
> 
> On 2003年7月28日 1:16PM -0500, Scott Cain wrote:
> > Hello,
> >
> > I would like to start assembling the first release candidate for GMOD.
> > This release would consist of chado, gbrowse, apollo and any necessary
> > scripts and other components to make it work well together. Among the
> > list of "things to make it work" are:
> >
> > - a script to load chado from genbank or GFF3
> > - scripts to allow Apollo to do the GAME-Chadx-chado tango
> > - documentation
> > - other things I no doubt haven't thought of yet
> >
> > I am working on the genbank/GFF3 loader. My question is, what is 
> > needed
> > to make Apollo do its thing? Is it (reasonably) well documented yet?
> >
> > Thanks,
> > Scott
> >
> > --
> > ------------------------------------------------------------------------
> > Scott Cain, Ph. D. 
> > ca...@cs...
> > GMOD Coordinator (http://www.gmod.org/) 
> > 216-392-3087
> > Cold Spring Harbor Laboratory
> >
> >
> >
> > -------------------------------------------------------
> > This SF.Net email sponsored by: Free pre-built ASP.NET sites including
> > Data Reports, E-commerce, Portals, and Forums are available now.
> > Download today and enter to win an XBOX or Visual Studio .NET.
> > http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01
> > _______________________________________________
> > Gmod-schema mailing list
> > Gmo...@li...
> > https://lists.sourceforge.net/lists/listinfo/gmod-schema
-- 
------------------------------------------------------------------------
Scott Cain, Ph. D. ca...@cs...
GMOD Coordinator (http://www.gmod.org/) 216-392-3087
Cold Spring Harbor Laboratory

Hi Mark,
I know the user docs for Apollo are very good--I was asking specifically
about docs for setting up Apollo to work with a generic installation of
chado. I suspect that they don't exist yet.
Scott
On Mon, 2003年07月28日 at 14:01, Mark Gibson wrote:
> Scott Cain wrote:
> > Hello,
> > 
> > I would like to start assembling the first release candidate for GMOD. 
> > This release would consist of chado, gbrowse, apollo and any necessary
> > scripts and other components to make it work well together. Among the
> > list of "things to make it work" are:
> > 
> > - a script to load chado from genbank or GFF3
> > - scripts to allow Apollo to do the GAME-Chadx-chado tango
> I think Colin or Chris can speak to this.
> > - documentation
> > - other things I no doubt haven't thought of yet
> > 
> > I am working on the genbank/GFF3 loader. My question is, what is needed
> > to make Apollo do its thing? Is it (reasonably) well documented yet?
> >
> 
> Scott,
> Apollo has good user documentation, as Nomi has done a lot of work on 
> the user guide. But unfortunately there is not a lot of developer 
> documentation. So are you asking about user or developer documentation 
> or both?
> Mark
> 
> > Thanks,
> > Scott
> > 
-- 
------------------------------------------------------------------------
Scott Cain, Ph. D. ca...@cs...
GMOD Coordinator (http://www.gmod.org/) 216-392-3087
Cold Spring Harbor Laboratory

Scott Cain wrote:
> Hello,
> 
> I would like to start assembling the first release candidate for GMOD. 
> This release would consist of chado, gbrowse, apollo and any necessary
> scripts and other components to make it work well together. Among the
> list of "things to make it work" are:
> 
> - a script to load chado from genbank or GFF3
> - scripts to allow Apollo to do the GAME-Chadx-chado tango
I think Colin or Chris can speak to this.
> - documentation
> - other things I no doubt haven't thought of yet
> 
> I am working on the genbank/GFF3 loader. My question is, what is needed
> to make Apollo do its thing? Is it (reasonably) well documented yet?
>
Scott,
Apollo has good user documentation, as Nomi has done a lot of work on 
the user guide. But unfortunately there is not a lot of developer 
documentation. So are you asking about user or developer documentation 
or both?
Mark
> Thanks,
> Scott
> 

I thought the parts of the first GMOD release were chado, gbrowse, and 
the web interface...?
Frank and Pinglei are coding the Apollo->chado cycle right now. Pinglei 
suggested we'd probably be looking at early- to mid- September before we 
could make a beta release available to GMOD.
Has the web interface been postponed?
-Dave
On 2003年7月28日 1:16PM -0500, Scott Cain wrote:
> Hello,
>
> I would like to start assembling the first release candidate for GMOD.
> This release would consist of chado, gbrowse, apollo and any necessary
> scripts and other components to make it work well together. Among the
> list of "things to make it work" are:
>
> - a script to load chado from genbank or GFF3
> - scripts to allow Apollo to do the GAME-Chadx-chado tango
> - documentation
> - other things I no doubt haven't thought of yet
>
> I am working on the genbank/GFF3 loader. My question is, what is 
> needed
> to make Apollo do its thing? Is it (reasonably) well documented yet?
>
> Thanks,
> Scott
>
> --
> ------------------------------------------------------------------------
> Scott Cain, Ph. D. 
> ca...@cs...
> GMOD Coordinator (http://www.gmod.org/) 
> 216-392-3087
> Cold Spring Harbor Laboratory
>
>
>
> -------------------------------------------------------
> This SF.Net email sponsored by: Free pre-built ASP.NET sites including
> Data Reports, E-commerce, Portals, and Forums are available now.
> Download today and enter to win an XBOX or Visual Studio .NET.
> http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_01/01
> _______________________________________________
> Gmod-schema mailing list
> Gmo...@li...
> https://lists.sourceforge.net/lists/listinfo/gmod-schema

Hello,
I would like to start assembling the first release candidate for GMOD. 
This release would consist of chado, gbrowse, apollo and any necessary
scripts and other components to make it work well together. Among the
list of "things to make it work" are:
- a script to load chado from genbank or GFF3
- scripts to allow Apollo to do the GAME-Chadx-chado tango
- documentation
- other things I no doubt haven't thought of yet
I am working on the genbank/GFF3 loader. My question is, what is needed
to make Apollo do its thing? Is it (reasonably) well documented yet?
Thanks,
Scott
-- 
------------------------------------------------------------------------
Scott Cain, Ph. D. ca...@cs...
GMOD Coordinator (http://www.gmod.org/) 216-392-3087
Cold Spring Harbor Laboratory

Hello,
I have somewhat of a conundrum: how to announce the creation of an
announcements mailing list? Since there is no good way to do it,
I'll just have to spam everyone I can think of. So, if you get this
message three or four times, my apologies.
Anyway, I've created a gmod-announce mailing list. This is a low
traffic, completely spam-free, moderated mailing list for the
announcement of software releases and meetings. Please subscribe at
http://sourceforge.net/mail/?group_id=27707 .
Thanks,
Scott
-- 
------------------------------------------------------------------------
Scott Cain, Ph. D. ca...@cs...
GMOD Coordinator (http://www.gmod.org/) 216-392-3087
Cold Spring Harbor Laboratory

Hi,
Just to confirm: the approved path for apollo->gbrowse is via the chado=20
database schema. We will probably have a GFF3 route in the not-to-distan=
t=20
future, but you can do what you want right now via chado.
Lincoln
On Thursday 24 July 2003 11:56 am, Michael Cipriano wrote:
> What I want to do is see what my options are for running gbrowse and
> using Apollo. Since the chado schema is part of GMOD, they will both be
> supported (so I think at least) by both of these applications. So I
> think, yes, that is what I want.
>
> Thanks for the response.
>
> Michael Cipriano (mci...@mb...)
> Research Assistant I, Global Infectious Diseases
> Josephine Bay Paul Center, Marine Biological Laboratory
> 7 MBL Street, Woods Hole, MA 02543-1015
>
> Michael Cipriano (mci...@mb...)
>
> Research Assistant I, Global Infectious Diseases
>
> Josephine Bay Paul Center, Marine Biological Laboratory
>
> 7 MBL Street, Woods Hole, MA 02543-1015
>
> Phone: 508-548-3705 (x6707)
>
>
> -----Original Message-----
> From: Scott Cain [mailto:ca...@cs...]
> Sent: Thursday, July 24, 2003 10:38 AM
> To: Michael Cipriano
> Cc: Gbrowse (E-mail); gmod schema
> Subject: Re: [Gmod-gbrowse] gff to gadfly schema
>
> Hello Michael,
>
> The short answer (as far as I know) is probably not. The longer answer
> is that, for GFF3, there will very soon be a loader into the chado
> schema, which is the schema that gadfly is migrating to soon. Any
> chance that is actually what you want?
>
> Scott
>
> On Thu, 2003年07月24日 at 10:59, Michael Cipriano wrote:
> > Is there a simple way of converting a gff file to something that can
>
> be
>
> > used to load into the gadfly schema?
> >
> > Thanks in advance.
> >
> > Michael Cipriano (mci...@mb...)
> > Research Assistant I, Global Infectious Diseases
> > Josephine Bay Paul Center, Marine Biological Laboratory
> > 7 MBL Street, Woods Hole, MA 02543-1015
> >
> >
> >
> >
> >
> > -------------------------------------------------------
> > This SF.Net email sponsored by: Free pre-built ASP.NET sites includin=
g
> > Data Reports, E-commerce, Portals, and Forums are available now.
> > Download today and enter to win an XBOX or Visual Studio .NET.
>
> http://aspnet.click-url.com/go/psa00100003ave/direct;at.aspnet_072303_0=
1
> /01
>
> > _______________________________________________
> > Gmod-gbrowse mailing list
> > Gmo...@li...
> > https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
--=20
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
Lincoln D. Stein Cold Spring Harbor Laboratory
ls...@cs...=09=09=09 Cold Spring Harbor, NY
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D

Nice!
Lincoln
On Wednesday 23 July 2003 04:46 pm, Scott Cain wrote:
> Hello,
>
> A few days ago, Dave wrote me to ask if I'd had success with partial
> indexes since I'd asked about it on the list a few weeks ago. As it
> turns out, while partial indexes frequently resulted in some improvemen=
t
> to query times, it was unreliable. Generally, poor query performance
> related to details in the query planner and which and when it used
> indexes on the feature table, a problem not directly related to the
> partial index anyway.
>
> The funny thing is, I had a solution to this problem that I worked on
> extensively a few months ago, but never rolled into the Das adaptor tha=
t
> I wrote for use with gbrowse: RTree indexes on featureloc. So, I added
> to schema/chado/modules/sequence/gff-bridge/sequence-gff-funcs.pgsql
> functions and an index on featureloc that I subsequently used in ROI
> queries for gbrowse. Now there is nice, consistent query performance o=
n
> ROI queries and gbrowse behaves well. See
> http://brie4.cshl.org:8000/cgi-bin/gbrowse?source=3Dchado for an exampl=
e
> of gbrowse running on top of the latest build of gadfly (v7). I believ=
e
> I included options for displaying most features for which there are
> feature locations.
>
> Scott
--=20
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
Lincoln D. Stein Cold Spring Harbor Laboratory
ls...@cs...=09=09=09 Cold Spring Harbor, NY
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D