Bug 177014 - History API design
Summary: History API design
Status: CLOSED FIXED
Alias: None
Product: ECF
Classification: RT
Component: ecf.core (show other bugs)
Version: unspecified   Edit
Hardware: All All
: P3 enhancement (vote)
Target Milestone: 2.0.0M4   Edit
Assignee: Danila Ermakov CLA
QA Contact:
URL:
Whiteboard:
Keywords: contributed, helpwanted
Depends on:
Blocks: 177020
  Show dependency tree
 
Reported: 2007-03-12 12:18 EDT by Danila Ermakov CLA
Modified: 2008-05-18 15:26 EDT (History)
3 users (show)

See Also:


Attachments
org.eclipse.ecf.presence patch (4.27 KB, patch)
2007-07-23 09:53 EDT, Danila Ermakov CLA
no flags Details | Diff
org.eclipse.ecf.provider.xmpp patch (4.61 KB, patch)
2007-07-23 09:53 EDT, Danila Ermakov CLA
no flags Details | Diff
history implementation plugin (10.52 KB, application/x-zip-compressed)
2007-07-23 09:56 EDT, Danila Ermakov CLA
no flags Details
Updated org.eclipse.ecf.history (20.33 KB, application/force-download)
2007-09-05 01:38 EDT, Scott Lewis CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Danila Ermakov CLA 2007-03-12 12:18:10 EDT
Following is a list of proposed functionality to be available thrue History API:
1. Possiblity to add/remove history entry (from, to, date, text) to repository
2. Possibility to retrieve entries by date range

Proposed design:

public interface HistoryManager {
	History getHistory(ID local, ID remote);
}

public interface History {
	HistoryLine[] getLines(Date from, Date to);
	void addLine(boolean incoming, String message, Date date);
	ID getLocal();
	ID getRemote();
}

public interface HistoryLine {
	Date getDate();
	boolean isIncoming();
	String getText();
}
Comment 1 Scott Lewis CLA 2007-03-12 12:34:48 EDT
Thanks Danila...looks good.

So I understand...the intention that this would be the chat history between two
individuals (as opposed to a chat room), right?

I would probably be inclined to put the access to the HistoryManager on the
org.eclipse.ecf.presence.im.IChatManager interface (e.g. via
IChatManager.getHistoryManager()).  Here are a few suggested changes to the API

public interface IHistoryManager {
        IHistory getHistory(ID chatPartner);
}

public interface IHistory {
        IHistoryLine[] getHistoryLines(Date historyStart, Date historyEnd);
        ID getChatPartner();
}

public interface IHistoryLine {
        Date getDate();
        boolean isIncoming();
        String getText();
}

How does this look to you?  I don't understand the addLine...isn't that implied
by the ability to retrieve history at all (from some chat service?).
Comment 2 Danila Ermakov CLA 2007-03-13 09:14:14 EDT
(In reply to comment #1)
1. IChatManager.getHistoryManager() - history is provider-independent (actually it is dependent indirectly - via chat participant IDs) resource. Having each provider implementing IChatManager I don't see why you need this getter.
2. Local ID - yes, there is no need in this info if we are talking about showing history in chat window. I thought it might be usefull in case of more complicated history viewer (ability to see history separated by sender IDs). Anyway - it can be added lately.
3. addLine - without possibility to add provider-independent send/receive listener I used it to add history entries from PresenceContainerUI.
Comment 3 Scott Lewis CLA 2007-03-20 18:06:14 EDT
See new interfaces IHistoryManager, IHistory, IHistoryLine here

http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.ecf/plugins/org.eclipse.ecf.presence/src/org/eclipse/ecf/presence/im/?root=Technology_Project

There are skeleton implementations present in existing providers, but none of them work yet.  Will plan to do implementation for XMPP first, but 
can't commit to doing it right away.  If others can contribute to implementation in XMPPChatManager (org.eclipse.ecf.provider.xmpp).
Comment 4 Danila Ermakov CLA 2007-03-21 08:16:27 EDT
I'd be glad to contribute (actually I already have implementation of it's functionality with XML storage for each partner) but I can't get following:
1. Why do you want to make it provider-dependent? To avoid common message notification? Having chat UI provider-independent it's not clear for me why UI should ask provider for history. 
2. What is purpose of Map of options?
Also I thought about possibility to plug in another implementation (encrypted for ex) via extension points.
Comment 5 Scott Lewis CLA 2007-03-21 11:22:02 EDT
(In reply to comment #4)
> I'd be glad to contribute (actually I already have implementation of it's
> functionality with XML storage for each partner) but I can't get following:
> 1. Why do you want to make it provider-dependent? To avoid common message
> notification? Having chat UI provider-independent it's not clear for me why UI
> should ask provider for history. 

By provider-dependent do you mean that the provider is responsible for adding history lines? (as opposed to having it be done explicitly by an 'addLine' method).  Since the implementation of any of the interfaces in org.eclipse.ecf.presence is provider-dependent (i.e. the entry point adapter IPresenceContainerAdapter is implemented by some provider) it seemed to make sense to just continue on in that vein.

Note that it's possible that the history implementation can/could be used by multiple providers...e.g. the history impl could be just an internal dependency for providers that wish to use it...probably through a common plugin.

> 2. What is purpose of Map of options?

To allow for search options (e.g. filtering, encryption, max size, etc) in the future.

> Also I thought about possibility to plug in another implementation (encrypted
> for ex) via extension points.

Yes, I think this is a good idea. 


Comment 6 Danila Ermakov CLA 2007-03-21 11:47:06 EDT
(In reply to comment #5)
If you mean to use same implementation for all providers - why do you need IChatManager.getHistoryManager() method? HistoryAPI clients can take it directly from factory or ext.point...
Comment 7 Scott Lewis CLA 2007-03-21 13:57:32 EDT
(In reply to comment #6)
> (In reply to comment #5)
> If you mean to use same implementation for all providers - why do you need
> IChatManager.getHistoryManager() method? HistoryAPI clients can take it
> directly from factory or ext.point...
> 

I see...yes, I agree that the history API could be completely separate from the presence/im API...e.g. exposed as an extension point or factory or OSGi service.  Perhaps it will make sense to expose as such as well...we can/could add these things as well.

The reason that it makes sense to me as accessible via the IChatManager.getHistoryManager() is because it's the history of a specific chat.  That is, it's not 'history of any sort of communication', but rather history of IM/chat in particular that's being captured by this API (at least so far).  I have no objections to moving in the direction of generalizing things to provide for history of other sorts of communication, etc...actually I think that's a very exciting direction, and a much more generic 'history API' for ECF would be very cool (aka ECF logging?).  But I thought that at least initially, given your initial enhancement request, it made sense to keep the scope relatively limited and give this API context by having it be a part of the presence API (which does define the context as 'im history').

Even having said that, I'm open to suggestions about alternative organization of the API as well.  The reason I created an IHistoryManager interface (rather than having getHistory(...) on the IChatManager) was that it doesn't seem impossible to me that the meta-operations on the history (e.g. setting the max history size, doing 'purges', etc) would need to be added, and the IHistoryManager would allow that...where adding such methods to IChatManager directly could result in a lot of API 'clutter'.  But it's a judgment call, and I'm open to other opinions.

Comment 8 Remy Suen CLA 2007-03-31 17:58:43 EDT
I think this API needs to be a little more abstract to support the logging of chatrooms.
Comment 9 Danila Ermakov CLA 2007-04-02 10:03:16 EDT
(In reply to comment #8)
Can PartnerID serve as identificator of chatroom?
Comment 10 Remy Suen CLA 2007-04-02 10:31:26 EDT
(In reply to comment #9)
> (In reply to comment #8)
> Can PartnerID serve as identificator of chatroom?

Well, a name change would be nice in that case then. Still, there is no way to identify as to who mentioned it since isIncoming() isn't very helpful when there are more than two parties in a chatroom.
Comment 11 Danila Ermakov CLA 2007-04-02 10:45:01 EDT
(In reply to comment #10)
Something like this?
interface IHistoryLineBase extends IAdaptable {
	public Date getDate();
	public String getText();
}
interface IHistoryLine extends IHisstoryLineBase {
	public boolean isIncoming();
}
interface IChatHistoryLine extends IHisstoryLineBase {
	public ID getSubPartnerID();
}
Comment 12 Remy Suen CLA 2007-04-02 12:59:24 EDT
Can't say I'm a big fan of this (sub)interface business. I think that IHistory should provide something like getConversationID() or getThreadID() or something of that nature. Then IHistoryLine should have a getSenderID() or something to replace isIncoming().

But maybe that's just me...
Comment 13 Scott Lewis CLA 2007-04-02 13:26:45 EDT
(In reply to comment #12)
> Can't say I'm a big fan of this (sub)interface business. I think that IHistory
> should provide something like getConversationID() or getThreadID() or something
> of that nature. Then IHistoryLine should have a getSenderID() or something to
> replace isIncoming().
> 
> But maybe that's just me...

Like many things, I think it's a matter of (API) taste.  I sort of like interface inheritance, as it does reflect that there are abstract history that are shared between both IM and chat room history, but I don't want to over use it and I think in this case it may not be needed.

How about we:

1) Remove getPartnerID() from IHistory.  Then IHistory has:

	public IHistoryLine[] getHistoryLines(Date start, Date end);

2) Remove isIncoming() from IHistoryLine

3)  Add two methods to IHistoryLine:

        public ID getSenderID();
        public ID getReceiverID();

For IM, 'incoming' messages will have getReceiverID().equals(user's ID).  For outgoing, getSenderID().equals(users's ID).

For chat rooms, getSenderID().equals(whoever sent message to chat room), and getReceiverID().equals(room id).

4) I move all the IHistory* interfaces to a new package...presence.history (peer package to presence.im and presence.chatroom).

What do you guys think of this?

Scott
Comment 14 Remy Suen CLA 2007-04-02 15:42:57 EDT
(In reply to comment #13)
What would IHistoryManager's getHistory(ID, Map) ID parameter's name be set to?

I am cool with the other changes above though.
Comment 15 Scott Lewis CLA 2007-04-02 16:02:30 EDT
(In reply to comment #14)
> (In reply to comment #13)
> What would IHistoryManager's getHistory(ID, Map) ID parameter's name be set to?

I thought it would be changed as follows.

	/**
	 * Get history for given targetID.
	 * 
	 * @param targetID
	 *            the ID of the target we want history for. May not be
	 *            <code>null</code>.  For IM/chat, the targetID should be
	 *            the <b>userID</b> of the user we are interested in getting
	 *            history for.  For chatrooms, the targetID should be 
	 *            the <b>roomID</b> of the chat room we would like history
	 *            for.
	 * @param options
	 *            any options associated with getting history info. May be
	 *            <code>null</code>.
	 * @return IHistory for given targetID. Will return <code>null</code> if
	 *         no history exists (with given options) for the given targetID.
	 */
	public IHistory getHistory(ID targetID, Map options);


Comments?

> 
> I am cool with the other changes above though.
> 

Comment 16 Remy Suen CLA 2007-04-02 16:19:56 EDT
Don't really like 'targetID' myself, but a parameter name change isn't an API issue if we do decide to change it later. ;)

Everything looks good to me, Scott. Comments, Danila?
Comment 17 Scott Lewis CLA 2007-04-02 16:31:01 EDT
(In reply to comment #16)
> Don't really like 'targetID' myself, but a parameter name change isn't an API
> issue if we do decide to change it later. ;)

Another possibility that I thought of...perhaps we could:

1) Remove IHistoryManager totally.
2) Add an IChatHistoryManager to im package
3) Add an IChatRoomHistoryManager to chatroom package

I actually now like this better.  Any thoughts?

Scott


> 
> Everything looks good to me, Scott. Comments, Danila?
> 

Comment 18 Scott Lewis CLA 2007-04-02 19:17:06 EDT
Changes to History checked in.  Did the following:

1) Remove getPartnerID() from IHistory.  Then IHistory has:

        public IHistoryLine[] getHistoryLines(Date start, Date end);

2) Remove isIncoming() from IHistoryLine

3)  Add two methods to IHistoryLine:

        public ID getSenderID();
        public ID getReceiverID();

For IM, 'incoming' messages will have getReceiverID().equals(user's ID).  For
outgoing, getSenderID().equals(users's ID).

For chat rooms, getSenderID().equals(whoever sent message to chat room), and
getReceiverID().equals(room id).

4) Changed documentation in IHistoryManager to read

	/**
	 * Get history for given targetID.
	 * 
	 * @param targetID
	 *            the ID of the targetID we want history for. May not be
	 *            <code>null</code>.  If being used for chat rooms, the
	 *            targetID is the <b>roomID</b> for the desired history.
	 *            if being used for IM/chat, the targetID is the ID of the
	 *            chat partner.
	 * @param options
	 *            any options associated with getting history info. May be
	 *            <code>null</code>.
	 * @return IHistory for given partnerID. Will return <code>null</code> if
	 *         no history exists (with given options) for the given targetID.
	 */
	public IHistory getHistory(ID targetID, Map options);

5) Added getHistoryManager() method to both IChatManager, and IChatRoomManager.

6) Added placeholder implementation of IHistoryManager to XMPP and IRC providers.
Comment 19 Scott Lewis CLA 2007-04-02 19:17:49 EDT
Opps.   This bug is still open
Comment 20 Remy Suen CLA 2007-04-07 19:49:39 EDT
I think we should introduce a mechanism for deleting the logs. Either a method that performs a full wipe or a full wipe and another one that wipes based on a date range.
Comment 21 Scott Lewis CLA 2007-04-07 21:00:57 EDT
(In reply to comment #20)
> I think we should introduce a mechanism for deleting the logs. Either a method
> that performs a full wipe or a full wipe and another one that wipes based on a
> date range.

Sure, seems reasonable.  Here's a possible API

In IHistory:

	public IHistoryLine[] clearHistoryLines(Date start, Date end);

If start is null, everything up to end is deleted, if end is null, everything from start is deleted, if both are null, then  everything for that IHistory is deleted.  The returned IHistoryLines are those that are deleted.

Another option is that we have another method on IHistory (and don't interpret null as described above...i.e. expect non-null Dates)

        public IHistoryLine[] clearHistoryLines();

Thoughts and opinions?




> 

Comment 22 Remy Suen CLA 2007-04-07 21:06:00 EDT
(In reply to comment #21)
I think I like #1 myself with the "double meaning".
Comment 23 Danila Ermakov CLA 2007-04-09 04:15:10 EDT
(In reply to comment #21)
looks good
Comment 24 Scott Lewis CLA 2007-04-09 11:10:32 EDT
    public IHistoryLine[] deleteHistoryLines(Date start, Date end);

Added to IHistory.

Comment 25 Remy Suen CLA 2007-04-21 10:23:15 EDT
I think this needs to be modified to provide some sort of API to grab a specific log or "instance". When you go take a look at some logs, it is generally split up by size, dates, or session. While the current API allows for the retrieval of a set of logs based on dates, it doesn't scale right when you want something like the most recent session or "last five lines".

For the former case, it is essentially impossible unless you know the proper Date to send in (which is probably impossible given the issue of milliseconds and all).

In the latter, you could technically go on forever iterating over getLines(Date, Date) by going back one day or one week at a time. Alternatively, if you send in null for both parameters, it could take a while for it to complete processing all of the logs.

Does this make sense?
Comment 26 Danila Ermakov CLA 2007-07-19 10:51:54 EDT
In process of adaptation of my old history implementation to current API I found it doesn't show how do we add new lines to the History. We need either HistoryManager.setChatManager to let HistoryMnager register listener in ChatManager it or History.addLine to add line to history from ChatManager
Comment 27 Scott Lewis CLA 2007-07-19 11:02:33 EDT
(In reply to comment #26)
> In process of adaptation of my old history implementation to current API I
> found it doesn't show how do we add new lines to the History. We need either
> HistoryManager.setChatManager to let HistoryMnager register listener in
> ChatManager it or History.addLine to add line to history from ChatManager
> 

I don't have any major resistance to adding such API, but I want to understand...what is the use case for arbitrarily adding lines to history?  Isn't this a form of revisionist history? :)

Comment 28 Danila Ermakov CLA 2007-07-19 11:18:15 EDT
> I don't have any major resistance to adding such API, but I want to
> understand...what is the use case for arbitrarily adding lines to history? 
> Isn't this a form of revisionist history? :)
if nobody add lines to the history how can something appear in there? I see following scenario of recording history:
1. HistoryManager listen to ChatManager events and record in/outgoing messages. In this case HistoryManager need to know wich ChatManager activates him.
2. ChatManager add outgoing messages to the history, after sending them, and add incoming messages after receiving fireChatMessage - need addHistoryLine

Comment 29 Scott Lewis CLA 2007-07-19 13:54:52 EDT
(In reply to comment #28)
> > I don't have any major resistance to adding such API, but I want to
> > understand...what is the use case for arbitrarily adding lines to history? 
> > Isn't this a form of revisionist history? :)
> if nobody add lines to the history how can something appear in there? 

Because they can be added implicitly by the provider impl (and/or the server that's behind it).  That is, when outgoing or incoming chat occurs it goes through the provider (via IChatMessageSender.sendChatMessage and IIMMessageListener.handleMessageEvent APIs).  The provider, in exposing these
APIs for sending/receiving simply records what happens (on client and/or server, depending upon protocol).

>I see
> following scenario of recording history:
> 1. HistoryManager listen to ChatManager events and record in/outgoing messages.

Right.

> In this case HistoryManager need to know wich ChatManager activates him.
> 2. ChatManager add outgoing messages to the history, after sending them, and
> add incoming messages after receiving fireChatMessage - need addHistoryLine
> 

But isn't this all in implementation (provider)?  I guess I still don't understand why the API has to expose this.  Sorry for being dense about this, I'm really just trying to understand.

Comment 30 Danila Ermakov CLA 2007-07-20 06:19:44 EDT
i got the idea - HistoryManager should look up available providers (IChatManager?) via extension points on statup, add IIMessageListeners and record everything it notified about. The only problem - according to code of XMPPChatManager it calls fireChatMessage only for incoming messages
Comment 31 Danila Ermakov CLA 2007-07-20 08:00:18 EDT
(In reply to comment #30)
an afterthought - in this case HistoryManager becomes singleton and method IHistory getHistory(ID targetID, Map options) doesn't have enough parameters to define IHistory - need IHistory getHistory(ID local, ID remote, Map options)
Comment 32 Danila Ermakov CLA 2007-07-20 08:55:27 EDT
(In reply to comment #31)
afterthought2 - another approach - each IChatManager has own IHistoryManager, then need kind of method IHistoryManager.setChatManager() to let it know where to register listener... or i can find it somehow?
Comment 33 Scott Lewis CLA 2007-07-20 12:21:29 EDT
(In reply to comment #32)
> (In reply to comment #31)
> afterthought2 - another approach - each IChatManager has own IHistoryManager,
> then need kind of method IHistoryManager.setChatManager() to let it know where
> to register listener... or i can find it somehow?
> 

Would be able to get the source for the relevant interfaces, modify them in your local workspace, and attach the modifications as patches to this bug?  Then we could apply the changes and look at the resulting API.  I think that would make it easier for me and everyone to understand what changes you are suggesting.  It's just hard to understand the implications of your suggestions when they are stated in prose.  OK?
Comment 34 Scott Lewis CLA 2007-07-22 20:26:07 EDT
Setting target milestone to 1.1.0.  Danila if you can respond to comment #33 please do.
Comment 35 Danila Ermakov CLA 2007-07-23 03:23:44 EDT
(In reply to comment #33)
agree - code explains better:)
will attach patch today or tomorrow
Comment 36 Danila Ermakov CLA 2007-07-23 09:53:20 EDT
Created attachment 74354 [details]
org.eclipse.ecf.presence patch
Comment 37 Danila Ermakov CLA 2007-07-23 09:53:51 EDT
Created attachment 74355 [details]
org.eclipse.ecf.provider.xmpp patch
Comment 38 Danila Ermakov CLA 2007-07-23 09:56:08 EDT
Created attachment 74356 [details]
history implementation plugin
Comment 39 Scott Lewis CLA 2007-09-04 20:22:50 EDT
(In reply to comment #38)
> Created an attachment (id=74356) [details]
> history implementation plugin
> 

I missed this on the 23rd...my apologies!  I'm getting the various pieces and will attempt to put them together over next few days in prep/hope for ECF 1.1 on Sept 7

thanks Danila for your contribution!
Comment 40 Scott Lewis CLA 2007-09-05 01:38:47 EDT
Created attachment 77680 [details]
Updated org.eclipse.ecf.history

Replaced org.eclipse.ecf.history plugin with revised version.

Changes made:

1) Removed references to IHistoryManagerFactory and historyManagerFactory extension point.  Although an extension point might be a good idea, I would like go ahead without the org.eclipse.ecf.presence.historyManagerFactory extension point for the moment.  Instead, provider implementation plugins can simply create a HistoryManager instance with the HistoryManager constructor (made public and in non-internal package in org.eclipse.ecf.history).  We may very well add the extension point back in later, but I don't see any problem with static dependencies on org.eclipse.ecf.history classes.

2) Restructured and simplified dependencies for org.eclipse.ecf.history plugin.  Removed dependencies on org.eclipse.core.runtime so that the plugin would work in server environment.

3) Changed some methods in HistoryManager class.  For example, the getLocalID() returned null and was called with getLocalID().getName().

4) Added some methods in HistoryManager class (e.g. getHistoryKey, getFileHistoryRoot).  Please look at these and notes in // XXX

5) Moved most classes in org.eclipse.ecf.history in to internal packages.

6) Removed org.eclipse.ui dependency (preference page) in o.e.e.history.  This was also to allow for usage in non-Eclipse Equinox environments (e.g. servers).

I would like to see if there was some other way to persist the history info than by putting it as a file on the file system.  Would it be possible to use the preference store in Equinox (e.g. user scope?).  Danila what do you think abou that?

Thanks again for the contribution Danila.  We'll get these things sorted out.

Also, another question:  all of the source files will need a copyright notice.  What copyright notice would you like to put in the files in org.eclipse.ecf.history?
Comment 41 Scott Lewis CLA 2007-10-16 15:11:07 EDT
Danila, do you have thoughts/comments about comment #40?  To move forward, need solutions to the questions.  Thanks.
Comment 42 Scott Lewis CLA 2007-10-16 15:11:28 EDT
Moving target milestone
Comment 43 Scott Lewis CLA 2008-04-18 01:18:52 EDT
History API added as of 1.1.  Implementation (bug 177020) not yet added.  Resolving this bug (API) as fixed.
Comment 44 Scott Lewis CLA 2008-05-18 15:26:20 EDT
closing