p2p.wrox.com Forums

p2p.wrox.com Forums (http://p2p.wrox.com/index.php)
-   Forum and Wrox.com Feedback (http://p2p.wrox.com/forumdisplay.php?f=56)
-   -   Reply E-mail System Requirements (http://p2p.wrox.com/showthread.php?t=644)

Hal Levy June 17th, 2003 10:03 AM

Reply E-mail System Requirements
 
Requirements are not negotiable.. We need to meet these completely to implement an e-mail solution.

1. Users posting messages must be validated against the subscription database. Validation must use some kind of information OTHER than the FROM: address. This means a posting password or some kind of digital signature to prevent against e-mail spoofing.

2. Headers that are "extraneous" must be stripped from postings.

3. Detection and removal of "overquoting" should be pretty foolproof.

4. The system must resist spam well.

5. The system must recognize out of office replies and derail storms of these.

6. Some companies send delivery notices for every e-mail received- these must also be prevented from getting to the list.

7. The system must intelligently handle bounces

8 The system must integrate with existing mail systems (MSSMTP or Lyris SMTP)

The system runs on IIS/SQL2K. The machines are clustered. Running ASP 3.0 (.NET is ok, with justification)



Hal Levy
Daddyshome, LLC
NOT a Wiley/Wrox Employee

Jeff Mason June 17th, 2003 12:47 PM

Boy, I don't know, Hal.

Starting off by stating that the requirements are not negotiable doesn't exactly present a tone conducive to constructive comments. If there is no possibility of give-and-take, what's the point of commenting?

I'll comment anyway. :D
Quote:

quote:1. Users posting messages must be validated against the subscription database. Validation must use some kind of information OTHER than the FROM: address. This means a posting password or some kind of digital signature to prevent against e-mail spoofing.
I'd be very curious to know what you envision as a workable implementation of this. I certainly agree that only registered users should be allowed to post to the forum via email. I can't see, though, a workable way to implement "...a posting password or some kind of digital signature ..." and still allow me to use the reply feature of my email client.

I only want to interact with the forum via email for day-to-day message posting and response to forums I have subscribed to. I'll go to the web page now and then for profile maintenance or to poke around in forums I have not subscribed to, but for the most part I want to use email as the primary means of communication with the forum. I want to receive a posted forum message via email. When I decide to respond to that message, from within my email client I simply want to click reply, quote the original as appropriate, add my comments and click send. If I want to start a new topic, I simply want to send a message to a specific email address and have the subject of my email be the topic subject.

If I have to add a password or other identifier, how would you propose I do it? Manually add another line to my post, or somehow insert a custom email X-header? (Can you even do that with Outlook? - too many of us use Outlook I fear) I don't see manually adding a password or digital signature line as a workable solution. I'll forget 50% of the time, if not more. Adding custom code to the email client won't work, either, as there are too many different clients in use, and no doubt some corporate rules would preclude 3rd party modification of a user's email client.

I just want to respond to the post, and if I have to go through hoops to do so, I won't, or more likely I'll simply forget - the email will (presumably) bounce, and I'll just give up.

I still would like to see evidence or testimony that email spoofing ever was a problem on the old email list.

I think Wiley (and now, you) is blowing this potential problem all out of proportion.
Quote:

quote:2. Headers that are "extraneous" must be stripped from postings
What's your definition of an 'extraneous' header? You are talking about email headers aren't you? They don't display anyway, so who cares?
Quote:

quote:3. Detection and removal of "overquoting" should be pretty foolproof.
I think limiting quoting to only the immediately prior message is appropriate. How you'll detect that I don't know, given various quoting styles (top versus bottom quoting), HTML vs plain text, etc.
Quote:

quote:4. The system must resist spam well.
What's spam? Nobody's figured out a foolproof or even workable way to detect it. If you have, what are you doing here? - go make a zillion dollars selling your solution ;)
Quote:

quote:5. The system must recognize out of office replies and derail storms of these.
Out-of-office replies are indeed annoying. I welcome any way you can to suppress them. I've never seen a "storm".
Quote:

quote:6. Some companies send delivery notices for every e-mail received- these must also be prevented from getting to the list
I've never received such a thing - I'm not sure what you are talking about here. Do you mean that some places actually acknowledge an email that I send to one of their addresses? Why would anybody do that?
Quote:

quote:7. The system must intelligently handle bounces
Bounces should be dropped on the floor, and perhaps the user to whom the message is sent should be automatically disabled from being sent any more email after a few (very few) such bounces.
Quote:

quote:8. The system must integrate with existing mail systems (MSSMTP or Lyris SMTP)
List managers already integrate nicely with existing email systems, you know... Tell me again why we are reinventing the wheel here?

Jeff Mason
Custom Apps, Inc.
www.custom-apps.com

Dan Jallits June 17th, 2003 04:13 PM

maybe it is late in the day for me Jeff (16:08 CST) or I haven't had my usually mix of Venti whatever's and Code Red's, but that last post seemed pretty sarcastic. Please disregard if I am wrong, but let's try to keep this a nice place

Best Regards,
Dan Jallits

Jeff Mason June 17th, 2003 04:38 PM

If my post comes across as sarcastic, I apologize to the community and especially to Hal. That was certainly not my intent at all.

I honestly don't understand some of the requirements, and I honestly don't know how some of them might be met.

I'm truly sorry if Hal, you, or anyone else interprets my questions and concerns as sarcasm; they certainly were not intended to be.



Jeff Mason
Custom Apps, Inc.
www.custom-apps.com

Hal Levy June 17th, 2003 06:30 PM

Jeff,

I am repeating what I have been told. Wiley is unwilling to have an e-mail system that does not meet ALL of these requirements and I am told these are not negotiable. I have collected their requirements and posted them here. I am not responsible for, nor do I necessarily agree with any or all of the requirements.


To answer your specific questions:

1. I don't know how we could implement the security the way Wiley is requiring it. Adding a line with a password perhaps. Or a PGP signature both would work, however are very invasive. I am looking for ideas from the community on how to do it. Wiley is not concerned with if it was a problem in the past- (this is what I mean by not negotiable) they want this functionality.

2. Yes, headers do appear in the e-mail when it's posted on the web site- we must filter them so they don't appear on the web boards. The web boards are not going away.

3. Exactly. I need ideas from the community.

4. I said resist- it doesn't have to be foolproof. But it needs to be able to do a fairly decent job of it.

5. Classic P2P had ACTIVE moderation- that's how come you never saw the storms (or the spam). They were blocked by the moderator(s). A storm is caused when Out of Office replies respond to Out of Office replies .. And so on and so forth...

6. I am told that many .GOV sites reply with a "receipt" ack for every message that comes into the domain. I have not experienced this- however Wiley is concerned about the problem.

7. Yes, and how is all that done? We are building a system from scratch here :)

8. Because the list managers don't meet all the requirements (1-7) plus the requirements for the web package. You and I may not care about the web interface- however, hit's on the old P2P show that it WAS used quite frequently.



Hal Levy
Daddyshome, LLC
NOT a Wiley/Wrox Employee

KenSchaefer June 18th, 2003 03:05 AM

Hi Hal,
WRT to point 5 - there are very few mail servers these days that result in "OOO" storms. Even everybody's favourite whipping post, MS Exchange, only sends 1 OOO to each "from:" address, and generally not to the list address (but rather the from: address).

WRT to the last comment "list managers" don't meet all the requirements, I'm interested to know which ones don't...

My experience of both Lyris and LSoft was that:
1) you could configure it so that each person had to use a password to confirm each message -or- an admin could approve each message

2) taken care of automatically

3) doesn't do this - whatever the user posts is included, but moderators can edit user posts

4) Both resist spam well, even if you just force the "from:" to be from a list members registered email address (coupled with moderation would eliminate all spam)

5) Addressed above

6) Those people should be unsubscribed from the list

7) Both LSoft and Lyris do this well. Lyris allows "x" number of bounces in "y" days before unsubscribing someone. Bounces never make it to the lists.

Lastly, Lyris can run off an SQL Server database, allowing you to build you own custom *web interface* to the list stuff, which is probably easier than trying to reinvent the wheel with-the-respect to building a listserver system... :-)

Cheers
Ken

www.adOpenStatic.com

KenSchaefer June 18th, 2003 03:06 AM

I should just clarify my previous post. I was just trying to say that both LSoft and Lyris pretty much address the stated concerns, and if you really want to build a similar system, it's probably best off looking at how those two systems work as a starting point.

Cheers
Ken

www.adOpenStatic.com

Hal Levy June 18th, 2003 07:04 AM

Everyone,

As I said, This isn't negotiable from the Wiley standpoint.

Arguing why LSOFT or Lyris will meet their needs isn't going to get us anywhere. I have been told that they will not consider anything but a MOD to Snitz.

Yes, I agree that it would be *MUCH* easier to implement a web front end on Lyris than an e-mail system on Snitz. But this is what we have been given to work with. And- again- I was told Wiley refuses to consider changing to Lyris and developing a web interface.


Hal Levy
Daddyshome, LLC
NOT a Wiley/Wrox Employee

JSample June 18th, 2003 08:49 AM

I will offer a “very brief” reason for each of the constraints that we posed to Hal. You don’t have to agree that they are good reasons, but in our mind they are valid.

1. SPAM was not an issue on the old site due to active moderation. Wiley does not have the dedicated resources to moderate a site that is generating hundreds postings a day after only a few weeks. With strong authentication we retain the ability to ban offenders, and make sure that if a user says something on the forum that there is no room for claiming someone else spoofed their account. For the future of the site we feel that SPAM is an ever increasing threat an needs to be addressed now rather than after it becomes a problem for everyone on the site.

2. Whether you have been affected yet or not, SPAMMERS are using address spoofing more and more each day. In order to not aid spammers in anyway, we keep everyone’s e-mail address on the site a secret so they cannot be harvested. If you reply through e-mail and the header information has your e-mail address in it, then we are just inviting SPAMMERS to come and take your addresses for sending their messages later.

3. Overquoting is a minor annoyance in e-mail, but on a web forum it makes the messages almost un-readable. While there are many of you that would only use email given the opportunity, there are an equal amount that prefer to only use the website and keep their mailboxes clutter free. We need to make sure that in fixing the site for one group we don’t turn around and break it for the other.

4. I believe that constraint number one ensures that the site will be very SPAM resistant. Also when you and number 2, this site become a model for what every SPAMMER on the internet DOESN’T want to see.

5. Obviously this is a threat to everyone using the system, but also one that every list based system has solved for. Their solutions should be easily replicated.

6. As funny as it may sound this is a new “policy” that many .GOV and legal sites have adopted. Wiley sends out almost a million newsletters each week from etips.dummies.com and our Cliffnotes etips. We are seeing more and more replies that simply state “your email has been successfully delivered to someone@somewhere.gov” Obviously no one wants to see the forums or their mailboxes cluttered with these.

7. This is a fairly simple requirement but with the registration system requiring valid email address this should not be a major problem on the site.

8 This one is a simple support issue. MSSMTP and Lyris are the technologies that we currently support and have already invested in. If a solution is found that uses sendmail for example, we would have a problem supporting it properly.

I hope you can now see why we have made these constraints and hopefully assist Hal in finding a resolution to them.

Thank you,

James Sample
Director, IT-Infrastructure
Wiley Publishing, Inc.

David Cameron June 18th, 2003 07:16 PM

I'm going to jump in on this late.

Quote:

quote:3. Overquoting is a minor annoyance in e-mail, but on a web forum it makes the messages almost un-readable. While there are many of you that would only use email given the opportunity, there are an equal amount that prefer to only use the website and keep their mailboxes clutter free. We need to make sure that in fixing the site for one group we don’t turn around and break it for the other.
Can I say at this point that you have broken the site for one group already, those people who used the site as a mailing list.

Ss I see it is that Wiley is concerned mostly with the web interface. I was concerned with the mailing list side of it. From my viewpoint improving on the original wrox web interface is a good thing, but not at the cost of destroying the email interface.

Secondly I'd it appears that wiley is being quite inflexible. Solutions have been proposed (more than once) for the problems that have been suggested, and yet the same problems continue to get put forward as reasons for not moving to an email reply system. Take point 2 above. Ken has made the point that the lyris system removes these. Others have suggested regular expressions. ASP 3.0 supports regular expression search and replacement, so use that remove all email addresses in postings that are not enclosed in ]URL[.

The issue as I see it is that Wiley has made a decision not to provide an email interface. Period.

regards
David Cameron


All times are GMT -4. The time now is 10:34 AM.

Powered by vBulletin®
Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
© 2013 John Wiley & Sons, Inc.