An sql dream - thinking in terms of order AND sets at the same time

How would you like to create sequential lists in a query? Think of the
possibilities that this ability brings. You could actually explain and
illustrate how queries really work. It's like taking an x-ray of your
query! It's working with two types (tables and lists) for the price of
one :-)

SET-ing your query in ORDER
http://beyondsql.blogspot.com/2008/04/dataphor-set-ing-your- query-in-order.html

www.beyondsql.blogspot.com

Re: An sql dream - thinking in terms of order AND sets at the same

>> How would you like to create sequential lists in a query? Think of the possibilities that this ability brings. You could actually explain and illustrate how queries really work. <<

Gee, I guess parallelism isn't important when doing queries. That was
why we got away from sequential lists and went to sets.

Re: An sql dream - thinking in terms of order AND sets at the same

On Apr 10, 11:52 am, --CELKO-- wrote:
> >> How would you like to create sequential lists in a query? Think of the possibilities that this ability brings. You could actually explain and illustrate how queries really work. <<
>
> Gee, I guess parallelism isn't important when doing queries. That was
> why we got away from sequential lists and went to sets.

History is only history if you are aware of it, as opposed to making
it up. That the system-R team was so heavily influenced by the idea of
parallelism eludes me. Perhaps you are confused by the concept of a
pointer vs. a process. And the fact that someone is a couch potato
nibbling away does not qualify one to be a nutritionist in the same
way as someone practicing sql does not qualify one to represent
themselves as an expert in sets. Please don't confuse sql with
relational. You are invoking the argument of "how" as the driving
force not "what". I do believe you have things backwards!
If you really did a quick tour of relational concepts you would find
'types' at the center, such as the table type and the list type. While
relationally all types are created equal sql sells the idea that they
are not. Sql is the judgement that a table is a higher life form than
a list. Sql sells the idea that a list is the misguided attempt to
represent a table as if their characteristics were not clearly
different as well as their uses. Complete nonsense. A type of pseudo
intellectual
misogyny where women are replaced by lists:)
In what I have presented exactly "how" the list is formed is
completely irrelevant. What possible difference does it make if the
query is executed with parallelism? However it's processed the list
will be populated and will reflect a "sequential" nature. And may I
remind you that is what lists do. And the query will produce a list
whether 'you' like it or not!:) One can use detailed query plans and a
good dose of common sense to interpret them. To that absurd idea that
everything happens
at once think of this as the fork that lets the air out of that:)
You are aghast with the idea of the "cowboy coder" but apparently you
have a lot of sql hillbilly in you:)

best,
steve
www.beyondsql.blogspot.com

Re: An sql dream - thinking in terms of order AND sets at the same

>> History is only history if you are aware of it, as opposed to making it up. That the system-R team was so heavily influenced by the idea of parallelism eludes me. Perhaps you are confused by the concept of a pointer vs. a process. <<

This has nothing to do with SQL per se. Parallelism in a set-oriented
model goes back to Cantor and the idea of treating an infinite
collection as a whole rather than a process. All we had was the old
Sigma and Pi notations and the concept of the limit of a process. The
lack of a set concept in math is what made Hilbert's hotel a problem
in its day. Hilbert wanted to knock on every door to move the guests
around; Cantor just shouts down the hallway and tells everyone to do
it all at once :)

>> If you really did a quick tour of relational concepts you would find 'types' at the center, such as the table type and the list type. <<

I would call a set, list, array, etc. data structures rather than a
types. I might put Domains, which are built on "types with
constraints" at the center ..

>> SQL sells the idea that a list is the misguided attempt to represent a table as if their characteristics were not clearly different as well as their uses. Complete nonsense. <<

I agree; a list depends on operations (head(x), tail(x), cons(x,y),
etc.) that have no counterparts in a set model. Likewise, the UNION
of a list makes no sense. But I don't see SQL as getting confused on
this point; I see newbies who are writing 1950's magnetic tape file
systems in SQL. They just don't get it.

>> A type of pseudo intellectual misogyny where women are replaced by lists:) <<

That is a weird analogy, even for you.

>> What possible difference does it make if the query is executed with parallelism? <<

(1) Speed of execution would be the big one for most people.
(2) Access to data is a major difference. Consider the statement
"SELECT a, (a+b) AS c, (c+d) AS e FROM Foobar;" which will work just
fine in a left-to-right processing order. But if columns a, c and e
are retrieved and constructed all at once, then c is not available for
e to use. Hell, you can write "SELECT (a+b) AS c, (c+d) AS b FROM
Foobar;" and get a result in a left-to-right language!
(3) This parallelism is why IDENTITY and other auto-increment
"features" make no sense when you insert a set. Given a statement
like INSERT INTO Foobar SELECT a, b, c FROM floob;" which puts (n)
rows into Foobar, how do you determine their auto-numbering? What do
you do when many sessions execute the same statement at once?

>> To that absurd idea that everything happens at once think of this as the fork that lets the air out of that:) You are aghast with the idea of the "cowboy coder" but apparently you have a lot of SQL hillbilly in you:) <<

That is changing fast. Have you ever worked with Teradata, WX2, SAND
or Vertica? The parallelism is at the hardware level. A little while
ago, someone put Postgres on a box with a solid state disk (I am
tracking down the article); it is all parallel processing and
associative memory access.

Re: An sql dream - thinking in terms of order AND sets at the same time

>> I would call a set, list, array, etc. data structures rather than a
>> types.

That would be a much lower level of abstraction. The term data structure has
connotations to storage, manipulation algorithms and efficiency all at a
much lower implementation level while types are at a higher logical level.

>> I might put Domains, which are built on "types with constraints" at the
>> center ..

Not sure if such a distinction is any useful since a type, by its very
definition, includes constraints and operations.

--
Anith

Re: An sql dream - thinking in terms of order AND sets at the same

>> The term data structure has connotations to storage, manipulation algorithms and efficiency all at a much lower implementation level while types are at a higher logical level. <<

Unh? You would consider an array of integers to be at a lower level
of abstraction then a mere integer? That would mean that scalar
arithmetic has to depend on array arithmetic, etc. Explain. I would
model this with an integer type with rules and operators, then build
integer arrays on top that model, extending the rules and operators
with access rules from the data structure.

For example, an additive operation, say +, on a 2D array that produces
a result R[i,j] is defined by (FOR ALL R[i,j] := (A[i,j] + B[i,j]))
where + is the scalar version from which we are building the
structured version.

Re: An sql dream - thinking in terms of order AND sets at the same time

>> Unh? You would consider an array of integers to be at a lower level of
>> abstraction then a mere integer?

No, I said the term *data stucture* implies aspects of implementation.
Therefore, at the logical level, it makes more sense to consider arrays and
lists as datatypes than to treat them as data structures.

>> I would model this with an integer type with rules and operators, then
>> build integer arrays on top that model, extending the rules and operators
>> with access rules from the data structure.

Well, if you have had built-in array types and list types, you wouldn't have
to build or extend anything. Even otherwise, just because an array or any
other seemingly complex type is being built using other seemingly simpler
types, wouldn't change it from being a type at all.

--
Anith

Re: An sql dream - thinking in terms of order AND sets at the same

>> No, I said the term *data structure* implies aspects of implementation. <<

So, when I use the term "array", you automatically forced into a row-
major physical implementation, instead of a column-major
implementation or or a linked list or a hash table or something else?
I seem to recall that PL/I and FORTRAN standards required different
row/column implementations, but we had arrays (matrix math) before we
had computers.

All I hear with *data structure* is an access method that can have
pretty much any data type at the node or cell level. I cannot see why
an access method could be a data type. The box is not the birthday
present :)

Re: An sql dream - thinking in terms of order AND sets at the same time

>> So, when I use the term "array", you automatically forced into a
>> row-major physical implementation, instead of a column-major
>> implementation or a linked list or a hash table or something else?

I doubt if it really a concern. The row-major order/column-major order
shouldn't be even relevant to user unless the language forces the user to
describe to how the array is traversed in memory and how each element is
accessed in memory. For single dimensional arrays, common operations like
CreateArray(n), GetElement(a,i), SetElement(a, i, ) doesn't
require specifying additional implementation level details. Obviously, if
performance is a concern then you'd think about the layout and storage in
memory, esp. if the array is of higher order, but that wouldn't be at the
logical level.

>> All I hear with *data structure* is an access method that can have pretty
>> much any data type at the node or cell level. I cannot see why an access
>> method could be a data type. The box is not the birthday present :)

If that is what you mean, then there is little contention. But I still think
that is too simplistic to be the well-understood description of data
structures in general.

--
Anith

Re: An sql dream - thinking in terms of order AND sets at the same

Joe said:
>I would call a set, list, array, etc. data structures rather than a types...

And from: ISO/ANSI: Introduction to OLAP functions
http://tinyurl.com/2taahc
"A window is named data *structure* associated with a SELECT block
(i.e., or )."

I suspect "structure" is a "standard" concept in sql. Now just what is
meant by "structure"? Can this "structure" be described in terms of
computer science a non rocket scientist could understand?:) From the
'Usual Suspects': "The best trick the devil played was convincing
everyone he didn't exist". My point is sql must have a computer
science of "structures" but it's a moot point. Sql has gotten along
quite nicely by hiding its computer science:) There's no reason users
need concern themselves with it. A SELECT doesn't return a type of
structure, it returns an "answer":) As far as sql and CS is concerned
there's no there there. I'd say that's quite a trick:) Besides when
you subtract the concept of a "variable" and a "type" I suspect it
becomes a little difficult to explain what the CS is. Does sql have at
its core something like the "quarks" of physics to explain
"structure"?:) This may explain the reason you wade into the
'implementation' pool, it's a lot easier:) Relational does have a
straightforward CS that does have consequences and can't be ignored by
the user. I'd say this does contrast with sql. Given that my example
concerning lists does go over the top I think it makes its point:)
Types are real and they have relationships with each other. And this
is independent of any way in which their implemented. But even though
relational has a CS and theory I can understand and that of sql eludes
me, who can deny sqls worth and use. Who can deny the idea of culture
and of pop culture:) Two different but substantial subjects.

When I said:
>> What possible difference does it make if the query is executed with parallelism? <<

I meant the list will be built one way or the other. Perhaps some type
of "ism" could influence how the order of rows are determined but when
the query completes you will have "a" sequential list. Perhaps you
would like the idea of a "black box" list better :)

>This parallelism is why IDENTITY and other auto-increment "features" make no sense when you insert a set.

Who says the IDENTITY value has to have meaning?:) I guess you also
like INSERT with IDENTITY..SELECT..ORDER BY. Now that is supposed to
have meaning. I owe it to everyone to write an article on that gem:)

best,
steve