Opened 7 years ago

Last modified 6 years ago

#4134 new defect

Canvas auto panning based on input focus

Reported by: erikos Owned by: garnacho
Priority: Normal Milestone: Unspecified
Component: Sugar Version: 0.97.x
Severity: Major Keywords:
Cc: Distribution/OS: OLPC
Bug Status: Unconfirmed

Attachments (7)

0003-OSK-im-osk-position.patch (13.1 KB) - added by garnacho 7 years ago.
last iteration of the gtk+ patch, meant to work on webkit too
0001-OSK-im-osk-position.patch (19.4 KB) - added by garnacho 7 years ago.
Updated OSK patch, scrolled window now works if hitting the document end
0002-textview-Update-handles-on-size_allocate-if-being-sh.patch (800 bytes) - added by garnacho 7 years ago.
textview patch, update handles on relocation after request-clear-area
0001-OSK-im-osk-position.2.patch (19.8 KB) - added by garnacho 7 years ago.
Updated patch, fixes some issues
0001-OSK-im-osk-position.3.patch (24.1 KB) - added by garnacho 7 years ago.
Updated patch
0001-OSK-im-osk-position.4.patch (24.0 KB) - added by garnacho 7 years ago.
Updated patch, has some necessary changes to implement Write logic
0001-Implement-OSK-resizing-on-the-AbiWidget.patch (3.1 KB) - added by garnacho 7 years ago.
Write patch, implement OSK resize in a subclass of Abi.Widget

Download all attachments as: .zip

Change History (30)

Changed 7 years ago by garnacho

last iteration of the gtk+ patch, meant to work on webkit too

comment:1 Changed 7 years ago by garnacho

This last patch deviates from the branch that's upstream. Unlike the patches there, it makes every IM-aware widget trigger upwards OSK-triggered resizes (including WebKitWebView and AbiWidget) with no additional changes, but does so by breaking the conceptual barrier between GtkIMContext and GtkWidget (the former isn't aware of the latter, mostly a design detail). This hack would only break (eg. not propagate resizes) silently if the IM context user GdkWindow does not pertain to a GtkWidget, AFAIK there's no such case

comment:2 Changed 7 years ago by erikos

I have two gtk3 rpms with the different patches, the one from the branch is http://dev.laptop.org/~erikos/gtk_rpms_os9/ and the second one at http://dev.laptop.org/~erikos/gtk_rpms_os9/osk2/

I have added a modified activity.py file from helloworld that has a Textview with some text. For debugging purposes there is as well the window.py from the toolkit-gtk3 with some debug output.

comment:3 Changed 7 years ago by erikos

Both patches do not work for me. When I click in an area that would be covered by the OSK, the scrolling is not done to keep that area visible. Some scrolling does happen but seems to be just due to the scrolling area itself.

Looking at the logs we do not receive the 'request-clear-area' signal, seen with both patches.

comment:5 Changed 7 years ago by erikos

Ok, I added the new rpm to my public repo, so it will be in the next build (os11). From a quick test, things get into the right direction.

We need to handle as well the case when we get to the bottom of a document. There has to be osk-height more space so you can get as well to the bottom. Not sure yet where this compensation code goes.

comment:6 Changed 7 years ago by erikos

comment:7 Changed 7 years ago by godiard

Tested in os11, this change do not solves all the issues.

We need resize the canvas area to use only the visible area, between the toolbar and the osk,
if not, all the content in the bottom will not be accessible. Can be seen in Terminal, Write or Paint.

Adding log in window.py request_clear_area_cb method, this method is never called, in Terminal or Write activities.
garnacho say: garnacho> gonzalo
, it's a signal that propagates from the bottommost to the topmost widget, other widgets may intercept it, eg. a scrolledwindow

I have tried catch the signal in other widgets without success.

I have tried to use the 'visibility-notify-change', on the canvas widget, and we can get a information about if the canvas is GDK_VISIBILITY_UNOBSCURED or GDK_VISIBILITY_PARTIAL, but I couldn't find how get the size of the real area obscured by the osk. Tried different ways, (the clipped area by cairo in the draw event, the allocated size by the widget) but all have the information like if the widget is visible behind the osk using all the available size.

I don't know if is available in any way, but is frustrating we don't have a simple api to solve simple questions like: is the osk visible? how much is the height/width? and commands to open/close on demand.

Changed 7 years ago by garnacho

Updated OSK patch, scrolled window now works if hitting the document end

Changed 7 years ago by garnacho

textview patch, update handles on relocation after request-clear-area

comment:8 follow-up: Changed 7 years ago by erikos

Thanks for the new patch Carlos. I did test with my modified HelloWorld activity, which has a scrolled window, textview inside. When I tap on text at the bottom the text to edit is moved up, right above the OSK. The cursor does as well update the position accordingly (it is over the OSK though).

If I hit enter, the current text gets under the OSK. Another issue is that if I scroll in that situation everything gets messed up in regards to the position. A regression for the scrolled window, is that when overshooting the widget does not take up the space again. A border is left.

The terminal and Browse use case both do not work with your patch. Is that expected. Anything we have to change here? Which use case do you test with?

comment:9 in reply to: ↑ 8 ; follow-up: Changed 7 years ago by garnacho

Replying to erikos:

If I hit enter, the current text gets under the OSK. Another issue is that if I scroll in that situation everything gets messed up in regards to the position. A regression for the scrolled window, is that when overshooting the widget does not take up the space again. A border is left.

I'm looking into fixing these

The terminal and Browse use case both do not work with your patch. Is that expected. Anything we have to change here? Which use case do you test with?

The VTE terminal widget has built-in scrolling and does not use GtkScrolledWindow, if Browse uses WebKit2 nowadays that might be the case too, I heard some months ago they were planning to do their own scrolling implementation

comment:10 Changed 7 years ago by dsd

We don't use WebKit2 yet because of its lacking touch support in the scrolled window. At the moment Browse uses a regular scrolled window.

comment:11 in reply to: ↑ 9 Changed 7 years ago by erikos

Replying to garnacho:

Replying to erikos:

If I hit enter, the current text gets under the OSK. Another issue is that if I scroll in that situation everything gets messed up in regards to the position. A regression for the scrolled window, is that when overshooting the widget does not take up the space again. A border is left.

I'm looking into fixing these

The terminal and Browse use case both do not work with your patch. Is that expected. Anything we have to change here? Which use case do you test with?

The VTE terminal widget has built-in scrolling and does not use GtkScrolledWindow, if Browse uses WebKit2 nowadays that might be the case too, I heard some months ago they were planning to do their own scrolling implementation

No, Browse uses WebKit. We do add the WebView to a scrolledwindow. In the activity window we do not get the 'request-clear-area' signal in both cases. I just removed the scrolledwindow in Browse, no signal neither.

Changed 7 years ago by garnacho

Updated patch, fixes some issues

comment:12 Changed 7 years ago by garnacho

There's a few issues I could see here:
1) The maliit IM context emits the osk signal far too many times, and on all imcontext around, not just on the focus widget's
2) Input method usage in the VTE terminal widget and WebKit don't fully behave like GtkTextView's or GtkEntry's, this was the reason the clear-request-area signal wasn't being emitted properly on Terminal or Browse
3) There are some intrinsic synchronicity issues on the handling of focus-in, button-press, button-release and request-clear-area, hence some funkyness when the scrolledwindow leaves place for the osk

The last patch I submitted does what as much as possible on 1, fixes 2 and partly fixes 3

comment:13 Changed 7 years ago by erikos

Thanks Carlos for investigating. I am building now with the new patch:

http://arm.koji.fedoraproject.org/koji/taskinfo?taskID=1256251

comment:14 Changed 7 years ago by erikos

comment:15 Changed 7 years ago by erikos

There are two cases in the shell where we want this to work: the Journal list view and the detail view.

In the Journal list view this is mostly working. When you tap on the title of an entry that is below the area where the OSK would cover the list does scroll up after a while. The title is half visible. This is as well the behaviour of items at the bottom of the scrolled window.

In the detail view we do not have a scrolled window. We could maybe add there one (that would only be visible and scroll when needed) to get this functionality.

Changed 7 years ago by garnacho

Updated patch

comment:16 Changed 7 years ago by garnacho

This patch solves multiple oddities in scrolledwindow, and has better interaction with WebKit webviews, there's still a bug in WebKit though where the cursor coordinates it reports to the IM context do not match with the coordinate space of the client GdkWindow given to the IM, so depending on the scrolling amount this works better or worse in Browse

Changed 7 years ago by garnacho

Updated patch, has some necessary changes to implement Write logic

Changed 7 years ago by garnacho

Write patch, implement OSK resize in a subclass of Abi.Widget

comment:17 Changed 7 years ago by garnacho

With these patches the textview and Write cases work nicely here, for Browse there still the WebKit bug where the coordinates don't match with the GdkWindow provided to the input method, looking into making a patch for this

comment:19 Changed 7 years ago by erikos

I did a new rpm and removed the patch from #4060 since that broke scrolling in general.

Write does still not work with the OSK. There is a black part below where the cursor is. Also the title editing in the Journal stopped working.

comment:20 Changed 6 years ago by dnarvaez

  • Component changed from sugar-toolkit-gtk3 to Sugar

comment:21 Changed 6 years ago by dnarvaez

  • Milestone changed from 0.98 to Unspecified

comment:22 Changed 6 years ago by dnarvaez

  • Bug Status changed from Assigned to Unconfirmed

comment:23 Changed 6 years ago by dnarvaez

  • Priority changed from Urgent to Normal
Note: See TracTickets for help on using tickets.