internet capable / internet ready OS

JdeBP> You are conflating TELNET the protocol with "telnet" the
JdeBP> applications program.

RP> No, I'm not.

JdeBP> [...] "raw e-mail" and "raw nntp" don't use the TELNET
JdeBP> protocol. They (most commonly) use NNTP, POP3,
JdeBP> and SMTP, layered on top of TCP, in turn layered on
JdeBP> top of IP. (Although they could also use QMTP,
JdeBP> MSOAP, IMAP, and so forth.) [...]

RP> This is true. But, some of those protocols are built on TELNET.
[...]

CK> And speaking for myself, also because I agree with
CK> Jonathon ('s facts).
CK> [...]
CK> TELNET makes no contribution to an SMTP session. The
CK> telnet application is kinda handy for setting up a TCP
CK> connection, and maybe even buffering keyboard input
CK> so we can correct mistakes before we send them. But
CK> the TELNET protocol is nowhere to be seen.

RP> [...]
RP> Again, that depends on the implementation. And, it
RP> doesn't change the facts that one can use a TELNET
RP> connection (probably due to MTP) and the fact that
RP> mail was built on FTP which uses the TELNET
RP> protocol. This is true even if SMTCP can be used
RP> on other protocols.
RP> [...]

CK> [...]
CK> And note that a TELNET connection intersperses control
CK> information with the data. My interpretation is that without
CK> this control information you don't have a TELNET
CK> connection, you have just a raw TCP connection. That
CK> interpretation underpins my entire thesis: TELNET has
CK> no contribution to make to SMTP.
CK>
CK> (I confess that I've digressed a bit and argued also that
CK> full TELNET is incompatible with SMTP, but that was to
CK> demonstrate that in order to use a TELNET connection
CK> with SMTP you have to reduce TELNET's contribution
CK> to zero. And in fact, on this point I'm speculating based
CK> on RFCs, not on experience.)

I, conversely, _am_ writing based upon experience, of writing clients
and servers for many of these protocols and of many other things.
Remember the "telnet" applications that I mentioned in an earlier
message, that only attempt to use the TELNET protocol when they are
connecting to that protocol's well-known port? I once had to use one
that _didn't_. (Step forward, IBM's "telnet" for OS/2!) It would, by
default, _always_ speak the TELNET protocol, irrespective of port
specified. Using it to perform SMTP conversations was tricky, because
of course the client would be speaking the TELNET protocol and the
server would be speaking the SMTP. The server wouldn't understand the
client, because they weren't speaking the same protocol, leading to
conversations such as this:

[C:\]telnet a.mx.yp.to smtp
220 a.mx.cr.yp.to NO UCE ESMTP
quit
502 unimplemented (#5.5.1)
quit
221 a.mx.cr.yp.to NO UCE
Session closed...

The SMTP server doesn't understand the first QUIT verb because the
"telnet" application is speaking the TELNET protocol, and has sent a
whole load of option negotiation characters over the connection before
sending the first 'q'.

By being the rare "telnet" application that always speaks the TELNET
protocol, unlike most other "telnet" applications, this application
serves as a reminder of the fact that applications don't map to
protocols. One may be using a "telnet" application to talk to a
server, but that doesn't mean that one is using the TELNET protocol in
order to do so.

CK> About the history... it doesn't matter whether once upon a time
CK> SMTP ran over FTP over TELNET, [...]

.... although it never actually did. FTP was a precursor mail
transport to SMTP, not a protocol that ever underlay it. SMTP isn't
and never has been layered over TELNET, either. SMTP is layered over
TCP for Internet purposes. Moreover, in practice the only other
things that it is commonly layered over are things such as an
operating system's internal stream IPC mechanisms (e.g. sending a file
to an SMTP server program, as Batch SMTP). As Mark Crispin once
explained, although SMTP was designed to work over protocols such as
NCP, it only ever reached the experimental stage in doing so. In the
main, SMTP was used over TCP over IP right from the get-go.

Re: internet capable / internet ready OS

On Thu, 25 Oct 2007, J de Boyne Pollard wrote:
> RP> This is true. But, some of those protocols are built on TELNET.

I do not know who "RP" is, but JdBP is correct and RP is wrong.

SMTP is most emphatically NOT built upon TELNET protocol. SMTP does NOT
have the distinguishing feature of TELNET protocol: the IAC negotiation
mechanism. You can NOT issue IACs to an SMTP session, and allowing TELNET
protocol over SMTP would break 8-bit SMTP.

However, an obscure bit of Internet trivia is that there IS a protocol,
other than TELNET, that is built on TELNET protocol. That protocol is
FTP! That capability has not been used for many many years, and very few
people still remember it. I've been around since the early 1970s and I
can never recall that capability actually being used.

What's more, there are numerous FTP server implementations that do not
recognize TELNET protocol. I'm probably the only person ever to have
noticed (since I just finished surveying some in the course of posting
this message). That doesn't mean that I care... ;-)

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

Re: internet capable / internet ready OS

On Thu, 25 Oct 2007, J de Boyne Pollard wrote:
> JdeBP> You are conflating TELNET the protocol with "telnet" the
> JdeBP> applications program.

JdeBP is correct on this detail as well. Most TELNET programs have
explicit code to disable TELNET protocol unless the connection is on port
23 or the server starts babbling TELNET protocol first. With the advent
of 8-bit character sets, the latter test ("the server starts babbling
TELNET protocol first") is often omitted since 8-bit characters can be
confused with TELNET protocol (especially so-called "old TELNET protocol"
which hopefully went extinct along with NCP in 1983).

Now, if RP (whomever he is) wishes to discuss these matters with me, he is
welcome to do so; but I would like to know his qualifications first.

I was around in the early 1970s. I wrote multiple implementations of the
early protocols in that time, including TELNET client and server
implementations for three different operating systems and one of the very
first SMTP implementations (said SMTP implementation was, for a while in
the 1980s, the leading SMTP implementation until UNIX and sendmail
displaced the systems in question).

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

Re: internet capable / internet ready OS

In , Mark Crispin writes:
>On Thu, 25 Oct 2007, J de Boyne Pollard wrote:

Sir Mark, you forgot to mention that you are also an
accomplished author. Your personal entry on Wikipedia
is a VERY fine example of self promotion.

later,
lin Baden


>Now, if RP (whomever he is) wishes to discuss these matters with me, he is
>welcome to do so; but I would like to know his qualifications first.
>
>I was around in the early 1970s. I wrote multiple implementations of the
>early protocols in that time, including TELNET client and server
>implementations for three different operating systems and one of the very
>first SMTP implementations (said SMTP implementation was, for a while in
>the 1980s, the leading SMTP implementation until UNIX and sendmail
>displaced the systems in question).
>
>-- Mark --
>
>http://panda.com/mrc
>Democracy is two wolves and a sheep deciding what to eat for lunch.
>Liberty is a well-armed sheep contesting the vote.



-----------------------------------------------------------
Let Ignorance and Intolerance Rule!

Baden Kudrenecky
baden@baden.nu
http://baden.nu/
-----------------------------------------------------------

Re: internet capable / internet ready OS

"Mark Crispin" wrote in message
news:alpine.OSX.0.9999.0710251014250.24162@pangtzu.panda.com ...

Follow-ups set to alt.os.development,comp.protocols.tcp-ip,comp.mail.misc.
Newserver restriction. The wide cross-post by JdBP was obviously to draw
you into the conversation... I believe I've picked the two groups to which
you post more frequently, yes?

> On Thu, 25 Oct 2007, J de Boyne Pollard wrote:
> > RP> This is true. But, some of those protocols are built on TELNET.
>
> I do not know who "RP" is, but JdBP is correct and RP is wrong.
>

I pity you for knowing Jonathan. He's proven himself wrong, illogical,
confused, and/or sidetracked on numerous topics, on dozens of NG's. If he's
correct, as you claim, this would be a miraculous first...

> SMTP is most emphatically NOT built upon TELNET protocol. SMTP does NOT
> have the distinguishing feature of TELNET protocol: the IAC negotiation
> mechanism. You can NOT issue IACs to an SMTP session, and allowing TELNET
> protocol over SMTP would break 8-bit SMTP.
>

As an individual who should be respected due to his contributions to various
protocols, I'm interested in hearing your explanation to the following. I
find it really odd you completely fail to make any distinction between the
control and data stream channels of the TELNET protocol. I.e., the
"distinguishing feature" of control channel negotiation doesn't encompass
the entire TELNET protocol, or even large portions of it. Jonathan de Boyne
Pollard and Ciarian Keating whose prior comments are on alt.os.development
(where an almost unrelated thread was usurped by JdBP), both seem to ignore
_all_ aspects of the data channel as well. Why are you of all people,
trivializing the importance and requirements of the data channel portion of
the TELNET protocol which was incorporated into other protcols? Your
trivialization seems to contradict the known historical facts. FTP (RFC959)
requires TELNET. MTP (RFC722) requires TELNET. SMTP(RFC821) requires
RFC822 (standardization of TELNET NVT-ASCII) from MTP.

> However, an obscure bit of Internet trivia is that there IS a protocol,
> other than TELNET, that is built on TELNET protocol. That protocol is
> FTP! That capability has not been used for many many years, and very few
> people still remember it. I've been around since the early 1970s and I
> can never recall that capability actually being used.
>

Yes, we already covered that and much more in the original thread...

In your other post:

> Now, if RP (whomever he is) wishes to discuss these matters with me, he is
welcome to do so; but I would like to know his qualifications first.

If you really weren't interested in discussing matters (implied by your
statement: "... but I would like to know his qualifications first."), it's
best not to post. Yet, you did so twice. Why incite, provoke, or slight
someone you don't know when you don't have to? That seems unwise...

If you're actually interested in discussing the matters with anyone, feel
free to posts your thoughts to the entire a.o.d. thread, after you've read
it in it's entirety, and not just the excerpts of context cross-posted by
JdBP.


Rod Pemberton

Re: internet capable / internet ready OS

On Thu, 25 Oct 2007, Rod Pemberton wrote:
> As an individual who should be respected due to his contributions to various
> protocols, I'm interested in hearing your explanation to the following. I
> find it really odd you completely fail to make any distinction between the
> control and data stream channels of the TELNET protocol.

There is no separate control and data stream channel in the TELNET
protocol. The TELNET protocol has control sequences inside the data
stream.

This lack of separation makes it particularly difficult for novice
programmers to implement TELNET correctly. It is easy for a careless
TELNET implementation to deadlock or get into a protocol loop.

> I.e., the
> "distinguishing feature" of control channel negotiation doesn't encompass
> the entire TELNET protocol, or even large portions of it.

Say what?

If you strip out TELNET control, all you have is a bidirectional
asynchronous data stream. That isn't TELNET. A three wire serial link
has that!

The TELNET specification dealt with the ICP negotiation (which is very
different between NCP and TCP), TELNET control, and what was called the
"network virtual terminal." All stream protocols do ICP; that's defined
by the transport layer (NCP or TCP) and the application layer only defines
the contact port. TELNET control and the network virtual terminal (the
majority of the TELNET specifications) are unique to TELNET, although FTP
is supposed to use them as well.

There is no "large portion" of the TELNET specification dealing with a
bidirectional data stream. That part is obvious, and as noted above is
the same as a three wire serial link.

> Why are you of all people,
> trivializing the importance and requirements of the data channel portion of
> the TELNET protocol which was incorporated into other protcols?

Because these is no such thing in the TELNET specification.

I think that your confusion is that you are ascribing to TELNET what is
actually defined by the transport layer: NCP (pre-1983) and since 1983 by
TCP.

You are correct in observing that not all application layer protocols work
that way. That's because they use alternative transport layers such as
UDP.

> Your
> trivialization seems to contradict the known historical facts. FTP (RFC959)
> requires TELNET.

Yes, but this is widely ignored today. I surveyed a number of FTP
servers, and many do not implement TELNET.

> MTP (RFC722) requires TELNET.

MTP was never deployed, and for excellent reasons.

> SMTP(RFC821) requires
> RFC822 (standardization of TELNET NVT-ASCII) from MTP.

That is as much of a stretch as Bill Clinton saying that he "didn't have
sex with that woman"!

There is nothing in RFC822 that refers to either TELNET or NVTs in any
way.

RFC822 does refer to ASCII, and the CRLF newline convention. But you
won't find the NVT requirement that a bare CR be followed by a NUL, other
than a passing mention in section 3.4.10 about network local conventions
in transport. However, neither TCP nor NCP has any such requirement, and
thus neither does RFC822 or SMTP. The CR NUL requirement is in TELNET
protocol, and presumably refers to some hack in which message are
transported via TELNET to a modem (actually did that sort of stuff in the
1980s).

> If you're actually interested in discussing the matters with anyone, feel
> free to posts your thoughts to the entire a.o.d. thread, after you've read
> it in it's entirety, and not just the excerpts of context cross-posted by
> JdBP.

I'd don't know the details of your quarrel with JdBP and prefer not to.
Nor am I taking JdBP's side; I'm simply noting that, given the text of
what he said and what you said that I saw, that he is correct and you are
not. That in no way states any opinion on correctness (or incorrectness)
of anything else that you (or he) may have said.

I am simply referring to the fact is that TELNET protocol is not the
foundation of other application-layer protocols, although it may have once
been intended that way. The experience with FTP was enough to get people
to realize that it was a bad idea; and as noted many FTP implementations
disregard that part of FTP.

The characteristics that you ascribe to TELNET are application-layer
protocol characteristics and not specifically TELNET characteristics.
NNTP and modern SMTP frequently transmits non-ASCII text in 8-bit; this is
not TELNET and in fact violates TELNET.


In conclusion, your problem seems to be one of nomenclature. We're
talking about the same thing, only you are (mistakenly) calling it TELNET
(an application layer protocol). This thing that we are discussion is
actually transport layer protocol, with some tentacles into session and
presentation layer; and in the modern Internet context is the TCP (as
opposed to UDP) protocol. Note that I said TCP, not TCP/IP. IP is a
network layer protocol.

A flame war when you are confused about nomenclature ultimately is a lost
flame war. Whatever valid points you may make are lost, as your opponent
will skewer you over the nomenclature. So it's really in your interest to
get the nomenclature right. You're talking about the characteristics of
TCP, and possibly also the Internet convention of CRLF for newline.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

Re: internet capable / internet ready OS

On Fri, 26 Oct 2007 10:52:21 +1000, Rod Pemberton
wrote:

> I find it really odd you completely fail to make any distinction between
> the control and data stream channels of the TELNET protocol. I.e., the
> "distinguishing feature" of control channel negotiation doesn't encompass
> the entire TELNET protocol, or even large portions of it. Jonathan de
> Boyne Pollard and Ciarian Keating whose prior comments are on
> alt.os.development
> (where an almost unrelated thread was usurped by JdBP), both seem to
> ignore _all_ aspects of the data channel as well.

Hi Rod,

(I'm not all that keen on having our mostly good-natured conversations in
a.o.d dragged into other newsgroups with very little accompanying context
like this, so I'll be brief here...)

I don't quite understand your remark about my seeming to ignore something
about the data channel. What is it that I'm ignoring? (A genuine question,
Rod, not a retort!)

Basically this is what I've been saying about the data/control channel
business:

In TELNET there is no control channel to speak of. OK, there's the DM and
perhaps one or two other control codes that get sent OOB, but the vast
majority of TELNET 8-bit control codes are sent in the data stream, and
it's up to the TELNET server to separate them. An SMTP server won't do
this, and so it will barf. As far as I can make out.


Cheers,
Ciaran

--
Ciaran Keating
Amadan Technologies

Re: internet capable / internet ready OS

> There is no separate control and data stream channel in the TELNET
> protocol. The TELNET protocol has control sequences inside the data
> stream.

Am I wrong that TELNET uses the TCP's OOB data?

--
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
maxim@storagecraft.com
http://www.storagecraft.com

Re: internet capable / internet ready OS

"Maxim S. Shatskih" wrote:
>> There is no separate control and data stream channel in the TELNET
>> protocol. The TELNET protocol has control sequences inside the data
>> stream.
>
> Am I wrong that TELNET uses the TCP's OOB data?

RFC 854 mentions use of Urgent signals within the TELNET protocol.

In implementations I've worked on in the distant past, one generally used
TCP OOB to handle the requirement. It's been a while and I'm too lazy to
study the RFC(s) to determine whether TCP OOB was specifically required or
an implementation choice(!)

Re: internet capable / internet ready OS

"Maxim S. Shatskih" writes:
> > There is no separate control and data stream channel in the TELNET
> > protocol. The TELNET protocol has control sequences inside the data
> > stream.
>
> Am I wrong that TELNET uses the TCP's OOB data?

TCP doesn't exactly have out-of-band data. Instead, it has an "URG"
pointer. _All_ bytes up to that marker (from the beginning of the
stream) are considered to be "urgent" for the application. There's at
most one urgent marker at a time, so if you send "urgent" again before
the receiver can read up to the old mark, the old marker disappears.

Yes, it's called "OOB" in the sockets interface, but that's not really
what it is. :-/ The only out-of-band aspect of this mechanism is the
signal ("hey! you have to read something now!"), and not the data
bytes themselves.

TELNET uses TCP's urgent data mechanism to deal with flush -- it's IAC
DM with URG set. See the "synch" signal described on page 9 of RFC
854.

--
James Carlson, Solaris Networking
Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084
MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677

Re: internet capable / internet ready OS

On Sat, 27 Oct 2007 07:34:35 +1000, Jim Logajan wrote:

> "Maxim S. Shatskih" wrote:
>>> There is no separate control and data stream channel in the TELNET
>>> protocol. The TELNET protocol has control sequences inside the data
>>> stream.
>>
>> Am I wrong that TELNET uses the TCP's OOB data?
>
> RFC 854 mentions use of Urgent signals within the TELNET protocol.
>
> In implementations I've worked on in the distant past, one generally used
> TCP OOB to handle the requirement. It's been a while and I'm too lazy to
> study the RFC(s) to determine whether TCP OOB was specifically required
> or
> an implementation choice(!)

RFC 854 (TELNET) states simply (and without emphasis) that TCP is the
underlying transport protocol. In its discussion of OOB/urgent data it
seems to waver between assuming and requiring that the TCP Urgent facility
is used for OOB data. Seems like a fair assumption to me.

But TELNET commands themselves are not sent OOB. From my reading of the
RFC, any TCP Urgent packet switches the server into a TELNET "urgent
mode", and a DM (Data Mark) command byte in the data stream switches it
back to normal mode. While in urgent mode, the server discards non-command
bytes.

A command byte can appear in the data stream at any time, regardless of
whether the server is in urgent mode or not. There is no correspondence
between "urgent mode" and any kind of "command mode" (there is no such
mode).

It doesn't seem to really matter much what data byte or bytes are sent in
the TCP Urgent packet - the RFC seems to assume that the server will
process OOB data inline. The RFC's discussion of its "synch mechanism"
does say that the DM byte should be sent in the Urgent packet, but that
seems to be incidental - as long as _something_ is sent OOB the synch
protocol will work. This is why I think there's an assumption that OOB
data is processed inline. Or have I missed something?


Cheers,
Ciaran

--
Ciaran Keating
Amadan Technologies

Re: internet capable / internet ready OS

On Fri, 26 Oct 2007, Ciaran Keating wrote:
> But TELNET commands themselves are not sent OOB. From my reading of the RFC,
> any TCP Urgent packet switches the server into a TELNET "urgent mode", and a
> DM (Data Mark) command byte in the data stream switches it back to normal
> mode. While in urgent mode, the server discards non-command bytes.

Speaking as one of the few TELNET implementors who actually implemented
all of this correctly, in actual practice this mechanism did not work very
well.

The problem was that TELNET client implementors had to worry about the
possibility of getting the DM before ever becoming aware of the "urgent"
state. It actually became worse in TCP than in NCP, even though TCP was
supposed to fix the problem.

What happened was that many TCPs implement(ed) OOB by delivering a signal
to the application as was done with NCP.

So, rather than risk output stopping waiting for a DM that would never
occur, the correct thing to do was to count DMs vs. urgent. At some point
in the future, if there were still more DMs than urgent, you might set the
count back to zero (meaning that you missed the signal entirely, perhaps
because two happened).

All of this was based upon a well-intentioned attempt to get output flush
(e.g., CONTROL/O on the old TOPS-20 systems) working so you wouldn't have
to wait for an extended period of time while huge amounts of buffered data
were still pooting on your terminal. Fortunately, the world improved and
we no longer use 1200 (or 300!) bps terminals!

I wonder how many modern TELNET implementations actually try to do
anything with urgent and DM.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.

Re: internet capable / internet ready OS

Mark Crispin wrote:

(snip)

> All of this was based upon a well-intentioned attempt to get output
> flush (e.g., CONTROL/O on the old TOPS-20 systems) working so you
> wouldn't have to wait for an extended period of time while huge amounts
> of buffered data were still pooting on your terminal. Fortunately, the
> world improved and we no longer use 1200 (or 300!) bps terminals!

Unfortunately networks got faster, TCP buffers got larger,
and programs displaying scrolling screens got slower.
In the old days, scrolling could be done by updating a
register in the display processor. Now, it is done by
copying many bytes of image data around in memory.

> I wonder how many modern TELNET implementations actually try to do
> anything with urgent and DM.

-- glen

Re: internet capable / internet ready OS

> In the old days, scrolling could be done by updating a
> register in the display processor. Now, it is done by
> copying many bytes of image data around in memory.

In videocard's memory, and usually this is done by a videochip and not the
CPU - the driver is smart enough to note that BitBlt runs with both src and dst
to be the video memory, and offloads this to a GPU.

Well, BitBlt is a Windows thing, but I think X11 has the analog.

This is so from good old S3 Trio PCI videochips from Pentium-166 era, if not
from Cirrus Logic 5428 from 486 era.

--
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
maxim@storagecraft.com
http://www.storagecraft.com

Re: internet capable / internet ready OS

In article ,
Maxim S. Shatskih wrote:

>Well, BitBlt is a Windows thing, but I think X11 has the analog.

No, bitblit is from 1975 or before and predates both Microsoft Windows. See
http://www.bitsavers.org/pdf/xerox/alto/BitBLT_Nov1975.pdf
http://en.wikipedia.org/wiki/Bit_blit
http://www.google.com/search?q=BitBlt+history


Vernon Schryver vjs@rhyolite.com