This compilation and all annotations are copyright © Jean Renard Ward, 1993, 2004, 2011, 2013, 2016.
I can be contacted at firstname.lastname@example.org.
History of Pen and Gesture Computing:
Annotated Bibliography in On-line Character Recognition, Pen Computing, Gesture User Interfaces and Tablet and Touch Computers
References from the approximate years 1994 to 1995.
Permission is granted to use this information in publication, including confidential reports, provided that accompanying text clearly makes reference to the URL for this page, along with the statement:
Source: Annotated Bibliography in On-line Character Recognition, Pen Computing, Gesture User Interfaces and Tablet and Touch Computers Copyright Jean Renard Ward
Most of these are in my personal collection, either as physical items or electronic files.
Copyright © Fri Jul 21 20:53:14 EDT 2017.
Mirrors of this page can be found at:
biblio93 (Prev page)
(Next page) biblio98
Vision of future technology. Shows personal hand-held video communication (similar to Skype) with pen touchscreen interface.
Implementation of a slot object: citations back to 1982.
Active Desk whiteboard / collaborative drawing space prototype using large Scriptel digitizer tablet and rear-projection display. Compare with Microsoft Surface, Digital Desk.
Add gestures (marks) to point-and-click direct manipulation Motif widgets without modifying Motif implementation: filter recognized gestures, callback hooks into mouse events, calls existing API function in widget.
Describes using a pen stylus for entering commands via character recognition / gestures into a computer, recognizing and executing command after out-of-proximity/end-of-stroke/time-out. Compare with Ledeen, which did not have gestures made on the display. Specific reference to first, second and third gestures in a sequence for a compound gesture (multiple flicks, multiple taps, etc.). Specific reference to direction of motion of the gesture. Animated marquee on selection after press-and-hold. Describes feedback to user (idealized form of gesture displayed, or question mark in a circle) on what gesture recognized. If gesture not appropriate in context, error returned (displayed). Proximity sensing for termination of gesture, highlighting of active objects (compare with tooltips).
Slider/selector/scrollbar for scrolling through long alphanumeric sorted lists (e.g. rolodex or dictionary): variable-speed scrolling (acceleration) if slider thumb moved more quickly. Acceleration not work well, especially since user could just tap on alphabetical index (linear alphabetically) next to scroll bar. Also micrometer (fine positioning) mode by moving mouse perpendicularly to scroll motion: confusing to users.
Optical (camera) hand tracking system for smart desk: track hand, multiple fingertips, finger joints.
IBM Simon: up/down physical buttons for volume on cell phone switched to contrast-adjustment mode either by pressing both buttons together, or tapping on a particular spot on the touchscreen. Debouncing of touchscreen. Both stylus and finger, interesting case of designing for fat-thumb with the use of a stylus (diplomatic statement about not losing the stylus).
IBM Simon: cellphone/smartphone with touchscreen, can show screens in both landscape and portrait orientation based on user on-screen rotation button.
Polhemus 3D digitizer/tablet position sensor: uses pulsed DC magnetic fields, so is affected by nearby metallic or ferrous objects. Compare with Ascension Technology?
Optical single-finger mousepad. Finger on flexible service deflects LED optical emitter onto four angular light sensors, amount of light sensed controls direction and speed of mouse motion.
Select new color from a palette, drag onto a UI element to change its color in OS/2. Dynamic themes/scheme manager via GUI.
File also contains EP published patent application 0 591 571 A1.
Video of Apple TabletBook, prototype Apple MacIntosh slate-format tablet computer, touchscreen on display.
Programming reference for QuickDraw graphic on MacIntosh. Regions or maskBits used for masking -- any number of closed regions defined in drawing coordinates vs. bitmap coordinates. modes (RasterOp) on copyBits (BitBlt) include SrcOr, srcXor, srcCopy. Automatic dithering and highlight (background color replacement) in modes. Modes blend (average), sum (addPin, addOver).
User manual for apple MacIntosh power book. Includes section on keyboard shortcuts (e.g. double-click gesture to open a program icon), use of trackpad/mousepad/touchpad.
User manual for ResEdit, MacIntosh resource-file editor. Bitmaps, icons, dialogs, etc.
Apple Newton support for wireless communications, not on a single-vendor network.
User-interface guidelines for Apple Macintosh: compare with similar publications for Microsoft Windows. Mentions stylus pen (tablet) as equivalent pointing input device. Pointers/cursors for scroll bars and controls (arrow), drawing and re-sizing (crosshairs), selecting and inserting text (I-beam), selecting fields in spreadsheet/array (plus sign), wait (wristwatch clock). Pressing == press and hold? References to academic UI literature. Cites to Buxton, Schneiderman for direct manipulation. Also Cites MinskyMR84 (gestures).
Apple prototype products, publicly known but not marketed: W.A.L.T. Wizzy Active Lifestyle Telephone with stylus -- compare with IBM Simon (also developed with BellSouth); Apple videophone PDA; Apple PenLite / Macintosh PowerBook Duo Tablet computer (full tablet/touchscreen computer, not PDA).
Video demonstration of Apple Newton user interface, including "heavy mark" gesture. Video in electronic files: transcript fetched from http://guidebookgallery.org 2010. Text file has complete transcription of text. Zigzag gesture to rubout (gesture). Copy-and-paste by moving text to edge of (small) screen. press-and-hold gesture on Newton "heavy mark", with highlight. Press-and-hold highlight is to move text (12:50), different drag and tap-drag gestures for move and for copy. At 12:20 shows rub-out gesture recognized in inking/writing mode. 12:40 pop-up virtual keyboard.
File transfer / connection utility between Apple Newton and Apple MacIntosh. Also calendar Date Book synchronization. Network connection? Parallel cable?
Apple Newton: describes various gestures and UI features, including "heavy mark" selection gesture: press-and-hold in text? Describes rotate button to change screen orientation.
Apple Newton: whitepaper about product concepts for Apple Newton: Apple Newton fax/phone (desktop) with notetaking, Newton Map, fold-out three panel touchscreen pen computer, Newton touchscreen wristwatch (larger panel -- wrist-mounted computer), Newton classroom whiteboard, Newton family message center (Message pad hanging on refrigerator).
Indexing (tree based) of electronic-ink names of objects by applying hidden Markov Models HMM to match the handwritten characters one at a time, take best match. Matching handles mix of hand-printing and cursive forms of a name. Compare HMM with actual recognition?
Automatic indexing of electronic ink notes doing approximate ink matching (AIM) instead of recognition. One application is indexing of multimedia objects.
Second volume in "Graphic Gems", collection of papers: 2D Geometry and Algorithms, Image Processing, Frame Buffer Techniques, 3-D Geometry and Algorithms, Ray Tracing (transparent or translucent objects), Radiosity, Matrix Techniques, Numerical and Programming Techniques, Curves and Surfaces.
ITS from IBM Research as automatic generation/generator tool for user interfaces. Contains Case study C: Wang Freestyle, pen-based voice/electronic-ink annotation system. Describes substantial public interest and acclaim when Freestyle released in 1988 (citations). See also individual articles.
Editing of (static/scanned) text images similarly to text editing, with word-wrap, insertion of other exemplars of characters from same document, searches of OCR-ed text. Notes that document image files may be longer-lasting formats than data files of documents.
Essay on legal, cultural, and other (non-technical) aspects of copyright protection and intellectual property, in the contest of DRM digital rights management.
Transfer function for isometric (non-moving) joystick pointing where effect of actual force is amplified non-linearly. Used in IBM TrackPoint II in-keyboard joystick. Asserts user performance comparable to mouse (linear mouse? non-linear?). No comparison with tablet.
Transparent menus/windows in graphical user interface, does not block view of elements on screen underneath.
To see windows underneath/obscured by other windows, user selects a mode whereby all windows become transparent: only the title bar and the frames are drawn. User can click on title bar (or frame) to bring that windows to the top, and transparency dropped.
Sketch editing (not sketch recognition), allowing refinement of drawn lines by user re-shaping spline fit of drawn curve with light overwrite marks/gestures.
Infinite resolution multimedia sketchpad: infinite resolution display windowed into smaller display area: parallel lazy loading, only load/render data that is current viewable: example is browsing photos.
Alternative graphical user interface to icons menus and windows, show small snapshots / postage-stamps of simulated desk objects. Directory browser is array of small thumbnail images of pages/documents.
Multiple scale / multi-scale data visualization user interface: example is directory browser, with thumbnails inside of thumbnails and multiple levels.
Zoomable user interface, example is "world view" of documents, similar to thumbnail. Compare with Telesis "world view".
Multiple graphical representations of data, constraint-based graphical editing.
Screenshots of Microsoft touchscreen PDA user interface, mostly from built-in tutorial. On-screen keyboard by touch of stylus to bottom-of-screen menu or an icon. Handwriting in boxed text fields: tap on field to bring up alternate word or character dictionary pick-list. Electronic ink stored as (erasable) bitmap image? Navigation tabs to right of screen -- compare with PenPoint? Screen rotation / orientation portrait/landscape chosen manually by user, similar to PenWindows dialog.
See-through transparent/translucent tools can be placed over an object without obscuring objects under the tool. One option is a gesture-interpreting tool. (Compare with gesture-buttons.) Two-handed input (trackball and mouse cursor: multitouch?) trackball moves a virtual transparent layer, such as a transparent selection/framing tool and a mouse click to make an action happen.
Video demonstration to accompany paper of same title. At about 2:00 shows dragging a "Fill Color" menu with transparent parts over an image. Dual-pane clipboard for picking up diverse multiple objects.
Video demonstration to accompany paper of same title. At about 0:25 shows a tool sheet (menu with buttons) with transparent sections being dragged over an image. Transparent pie menus / marking menu gestures. Unistroke gestures on a tool: compare with gesture buttons. 3:10 gesture interpreter tool / handwriting pad (Goldberg unistroke recognition). Mnemonic gestures: e.g. "C" for copy, "T" for translate/move --.
Video demonstration. Shows tool window with frame and buttons (but most is transparent) being dragged over a map, and certain features of the map are altered (pixels combined?) Also, at about 0:59 shows translucent overlaying of two tool windows.
Video demonstration. At about 0:50 shows window tool with circular "view" area that changes color of pixels underneath (combined pixels), with part of tool transparent, also with buttons and controls in the window.
Mathematical process for aligning/matching portions of surface from multiple views from a single camera. Compare with structure-through-motion.
PIM/Diary that keeps change/delta diff record of changed entries so that previous entries can be reconstructed, and automatic monitoring for dates and other phrases to prompt user to add to diary. Compare to marking menus/keyboard?
Mentions Scriptel WriteTouch front-mount digitizer for Windows95 and Windows for Pen Computing: cites Windows 95 as having better pen-computing support than Windows for Pen Computing 1.0, although tablet drivers are not included. Norand PEN*KEY6100 vertical market pen tablet device.
Tutorial on alpha blending / compositing / matting /overlaying (synonyms), mentions anti-aliasing as one use, primarily "over" operator for bitblt/raster-op from Porter and Duff.
Tutorial on alpha blending / compositing / matting /overlaying (synonyms). Focuses on optimizations of integer arithmetic for 8 and 16 bit integer operations on 8-bit (32-bit) RGBA values, rounding and quantization errors.
Capacitive matrix grid digitizer: objects near intersections of wires affect capacitance between wires. Gets area measurement of adjacent intersection points, computes centroid/peak from adjacent (independent) sensors nodes. Used for mouse and for touch keypad in one device. Compressibility of fingertip to infer pressure (force). multi-touch capable? Compare to Buxton 1985 multi-touch capacitive tablet. Press with greater force/pressure to drag.
See also Radio Drum, Boie 1984 paper on capacitive sensors, references on "capacitive moments".
Sealing for SAW touchscreen tablet: see citations for actual surface acoustic wave touchscreen technologies.
Elographics IntelliTouch Systems Manual, 1988.
Recommends PenRight! Pro as leading pen-centric development environment, Windows for Pen Computing lacks pen-centric features, non-pen-centric development tools are unfeasible. No mention of Newton or PenPoint.
Electronic ink text editor, electronic ink automatically segmented based on spacing without character recognition, gestures include lasso and brackets to select, arrow to move text, double-ended arrow to copy, caret to insert, X to delete. Compare with van Raamsdonck.
Sonically-enhanced control bars and other elements in GUI graphical user interfaces. Two simultaneous earcons (audio tones) useful for two parameters.
Auditory/sound feedback on scrollbar: auditory status display frees up screen space. Single tones for scroll events, high pitch for movement up, low pitch for movement down, separate sound for kangaroo jump (click above/below thumb to jump one page).
Earcons: Activate-on-lift behavior on buttons introduces problem that user may slip off button and not know it: add audible sound feedback.
Earcons: two-tone (musical) audio feedback perceived/recognized as well as sequential tone.
Hand-held touchscreen/PDA device with bar code reader/scanner for tracking package deliveries (think UPS or FedEx). Using stylus/pen near touchscreen disables finger-touch input (e.g. palm-rejection problem). Delayed transmission of data (compare with PenPoint?). Check-mark gesture to delete. Signature capture, but no authentication. Active battery-powered stylus with capacitive coupling. Inactivity timer/time-out to turn backlight off.
Smartphone / communicator with video camera and display: camera is mechanically steerable.
tablet-free stylus/pen with tip-force (pressure) sensor and accelerometers in tip, handwriting recognition (actually character recognition). Compare with Crane/SRI accelerometer pen (not cited). Uses special delimiter characters/gestures (e.g. right/left corners) to mark start/stop of recognition.
Web browser, pages can be stacked in multiple decks on screen: no 3-D effect on desks -- compare with Freestyle documents. Suggested use is keeping related web pages together in a single deck.
Optical digitizer tablet: Note similarity to Anoto: printed surface of pattern cells, making use of sub-patterns. Refers also to a "single-pixel" sensor stylus.
PenRight cross-platform development environment for pen computing: PenRight! for Windows create Windows and DOS applications for PDAs, tablets, handhelds, etc. from single source code.
Haptic feedback stylus/pen (one application is simulating surgical feel). Stylus tip connects to direct-coupled (no gears) actuators, 3 DOF degrees of freedom (not 6). Special attention to open-loop control functions.
See also http://www.billbuxton.com: User flips keyboard top over to expose touchscreen: touch tablet is multi-touch sensitive and pressure (force) sensitive, with virtual sliders and further touch elements.
Two overlapping operations by user as same time using pointing input in both hands. Describes scrolling gestures: smooth scrolling as touch-slide-release, page turning with touch-rapid flick-release, and jumping to scrollbar touch location on a touch surface. Also mentions wireless touch tablet PARCtab.
See also http://www.billbuxton.com.
Graphical on-screen keyboard using gestures for modifier keys such as shift and control: e.g. tap "a" key for lower-case a, upward flick for Upper case, right flick gesture for Control-a, right-upward angle gesture for control-shift-A, etc. Mentions interchangeability of other types of input devices: mouse, tablet, trackball, etc. press-and-wait / PenStill / press-and-hold for marking menu, gesture and wait at end of mark also for multiple hierarchy of menus. Four events: PenDown, PenDrag, PenUp, PenStill (with no movement for a period of time).
haptic input overview.
Date approximate based on dates of most references, this draft obtained 2002.
Video presentation of two-handed user interface with tool in non-dominant hand, tablet stylus in dominant hand. Companion video to "A taxonomy of see-through tools" and "Toolglass and magic lenses".
Object-based graphical development system with polymorphic inheritance.
Collection of tutorials on information/computer security. Personnel and administrative policies, legal aspects. Long treatment of cryptography/cryptology/PKI, basic techniques of virus/worm including hooking. Access control primarily about physical access / authentication of users.
Handwriting recognition that let use display pop-up list of alternative recognition results for words (from spelling dictionary) by entering a gesture to bring it up, user can pick appropriate word, or use virtual keyboard to type in correct word.
Rotate an object with a stylus by means of a center axle mark, and a handle used to rotate.
On pen computer or tablet, pop-up list of alternative handwriting recognition results, user can pick from list or enter word via virtual keyboard. Figures show pop-up keyboard of Apple Newton. Correction gestures (insert, delete gesture, etc.). Refers to keyboard gesture (tap on keyboard image), correction gesture (tap on menu of alternative words).
Stephen Capps is listed in over 50 patents for pen-computing user-interface features, mostly from the Apple Newton.
Gestures on buttons or command-input areas to edit electronic-ink notes: notes not necessarily ink.
Few if any references before. Compare with van Raamsdonk, Pencept, and CIC.
Automatic re-flow / formatting of paragraphs from handwritten (or "pointing" input): compare to van Raamsdonk.
Stephen Capps is listed in over 50 patents for pen-computing user-interface features, mostly from the Apple Newton.
Few references before 1990 Compare with van Raamsdonk, Pencept, CIC.
Stephen Capps is listed in over 50 patents for pen-computing user-interface features, mostly from the Apple Newton.
Patents cite little or no prior art before 1990 ... Compare with van Raamsdonk, Pencept, CIC.
Stephen Capps is listed in over 50 patents for pen-computing user-interface features, mostly from the Apple Newton.
Patents cite little or no prior art before 1990 ... Compare with van Raamsdonk, Pencept, CIC.
Shows graphical/touchscreen slide switches and toggles, three-state "secure switch" toggle. Compare to Apple slide-to-unlock gesture?
User Action Notation for user interface objects. Example: Secure switch graphical object for preventing accidental manipulation of a device under computer control, such as master power switch for factory machinery. Compare to Apple slide-to-unlock gesture?
Personal note on EO with PenPoint: lists of collected EO/PenPoint materials. EO 440 hardware specifications. EO based in UK?
Compare Pen-Windows and CIC Handwriter recognizers: the CIC handwriting recognizer performed better, both with and without constraints.
GRIGRI (French slang for "scribble") system using pen-computing gesture recognition in user interface for air traffic control ATC. Modeled on flight strips for air traffic control. See also "Digistrips". Numeric entries for flight numbers via menus. Cites PenPoint. Lists fifteen single-stroke gestures, some of which are alphabetic: set warning, lasso (select), C, erase/pigtail gesture, undo alpha gesture, h change heading, etc.. Mentions that pen-based computing is bad for large ATC display screens (arm fatigue). Refers to gesture input as "mark-based input". Uses semi-transparent windows /button so as not to obscure radar image underneath window. See Rockstar Samsung.
Graphical cut-and-paste for drawing flowcharts (visual programming languages): MacDraw "paint" style cut and paste awkward. Suggests semantic cut-and-paste that is specific to graphical objects -- automatically handle connection arrows, etc.
Summary and overview of physical types of targets used in 3D capture: e.g. LED's, patterned crosses, reflective films, etc. No mention of using features of existing objects.
Portable slate pen-computer using touch screen (not proximity digitizer): Microslate, Inc. of Canada. Describes having an automatic application generator. Refers to drop-down pick list as a "library" of answers for touch-screen selection.
Web posting by Liam Comerford on predictive (vocabulary) virtual keypad/keyboard in IBM Simon smartphone. Software in Simon used proof-of-concept vocabulary data, rather than product-quality data.
The Grunt: Texas Microsystems 3.5 pound Windows for Pen Computing portable computer using MicroTouch touch-screen: pen or special pen/stylus.
Pen-based computers / pen computers useful in data collecting with UPS, engineering maintenance on ships, of immediate access to data (stored in device) and faster to input test or delivery data. Mentions that generally no or little use of handwriting recognition, data entry by picking on pre-designed forms, adjusting sliders, sketches. PoquetPad hand-held pen computer, CruisePAD separate tablet/display for wireless access to PC within 100 meters.
Transparent overlay windows in graphical user interface, pixel blending.
Instead of password, user inputs a pattern of touches or events for authentication: slide-to-unlock?
Architectural overview of the PenWindows / Windows for Pen Computing system from Microsoft: hedit and bedit controls, Pen Extensions, normal and inking mode, RC handwriting recognition manager, PenWin.dll.
Visual video tracking of hands and fingers using simple correlation of a template. Template must be change with changes in finger orientation, image size. Cites to Wellner Digital Desk.
LabView manuals from early 1990's for visual programming language. Shows slide switches, manuals for sale?
twist/bending/flexing sensor using loops of light guides along a linear strip (see ShapeTap). Light guide is serrated on one side, bending of light guide results in change/loss of light transmission when light guide is bent. Angling the serration (different angles on two light guides) to sense twisting.
Suggest "X" windows as solution to lack of control over exact appearance of HTML documents: Pixel-level control. Extends to remote execution (X-client/server). Suggest LBX (Low Bandwidth X) protocol to reduce latency/bandwidth problems. Similar presentation to "X Over the Web", WWW3, Darmstadt, Germany April 1995.
Open Look windows management system / user interface. Drag-Right-Distance to select sub-menu. MoveThreshold, MouseDamping system properties. Compare with press-and-hold.
Electrostatic / capacitive digitizer from Scriptel, can sense finger input as well as stylus.
Attempt to study audio feedback (audible click) on virtual buttons with activate-on-lift (on mouse). Problem was user's rarely made errors. Experiment modified to require user to click on separate "accept" (virtual) button for every digit key/button entered.
Donald Drumm "Orb" project from Wang Freestyle: pointing device for accessibility, uses head-mounted tilt sensor to push cursor position, simple voice detection circuit that detects fricatives "s" sound versus vocalized sounds.
Accessibility on GUI graphical user interfaces improved for both blind/visual impaired and "visually distracted" users by combining multiple modes of feedback: tactile edges to buttons, press/depress haptics on buttons, auto signals. Hardware used audio plus mechanical pantograph with vertical force-feedback: texture could only be felt by user scanning stylus point over surface.
Standardization functional specification of Microsoft Windows 3.1 application programming interface. WS_EX_TRANSPARENT creates transparent window. Also GetBkMode TRANSPARENT parameter for drawing. Transparent windows are included in windows hit testing, transparent windows can be buttons/scrollbars. See Rockstar Samsung.
LabVIEW graphical programming language.
Graphical user interfaces problematic for blind/visually impaired. Solution is not screen reader, rather semantic-level interface for the blind. Auditory icons, spatial vs. hierarchical modeling of (user-interface) object relationships. Semantic vs. Syntactic/low-level presentation at different levels of underlying user interface API. Mercator on X-Windows translates underlying X toolkits, speech/auditory output (DECtalk, via network transport RPC): experimenting with tactile (Braille?) output.
Graphical user interfaces problematic for blind/visually impaired. Solution is not screen reader, rather semantic-level interface for the blind. Mercator on X-Windows translates underlying X toolkits, speech/auditory output: experimenting with tactile (Braille?) output.
Accessibility access to graphical user interfaces for blind/visually-impaired, construct and traverse hierarchical tree representation of user interface structure. Keyboard and voice interaction replace mouse.
Solid modeling tool for 3D models, uses sketch recognition of arcs, lines etc. with pen, gestures based on pen direction (pen/stylus better than mouse due to greater dexterity). Discussion of constraints in interpreting sketches.
Brief bio of John Elias: multitouch user interfaces in 1995, later founder of FingerWorks, acquired by Apple. See also Westerman.
Refers to mapping of events to mouse buttons in user interface. Compare to press-and-hold.
Contains description of touchscreen implementation of mouse menus for DOS application which are not "mouse-aware". See also Microsoft documentation on MENU.COM and mouse menus. Tablet drivers for mouse emulation in DOS and Windows.
Refers to reduced expectations on handwriting recognition accuracy, not to read human-illegible handwriting. Reference to Doonesbury commentary, use of point-and-tap selections and digital ink. Shows modified Graffiti character set (neography), Lexicus Cursive Recognition.
Review of products and the problems of collaborative on-line editing tools for editors and writers. Suggests sticking with "traditional" mark-up symbols: easily learned, familiar, well developed for the problem domain. States pop-up or embedded comments awkward. Change tracking does not do well showing extensive formatting changes to users. Recommends digital ink (electronic ink) with gesture commands (the mark-up symbols), but commands not executed until approved by author. PenEdit (Advanced Pen Technologies), MATE, Red Pencil.
Proposal to use a DataGlyph (two-dimensional barcode) to having indexable/scanable information at the start of published articles. Compare to Silverbrook patents of 2004, and to Anoto. Note: there was a similar technology for a 2D barcode date for articles already in place in the 1980's?
Creating of a "handwriting-looking" font from samples of someone's handwriting: refers in claims to automatic classification/categorization based on slope and other characteristics of the characters.
Travelling software program: interpretive code (P-code: compare mobile Java objects) moves from user to user to handle PKI cryptographic signing and verification. Includes code-signing of program segments, transmit the certificate segment ahead of the program.
Signed authorization tokens (program authorization information) stating what capabilities (rights/permissions) a program may exercise. Tokens may be co-signed by multiple permission-granting authorities. Include checking timestamp and signature of program getting permission. Compare with DRM tokens?
Magic Lens project: application to database systems -- e.g. user requests map with information (from database) as to properties to rent. Magic Lens "filter" in window of GUI tool to change how information is display / what information is displayed.
GraspDraw project: Physical devices coupled to "virtual" objects via position sensors: blocks (bricks) that may be moved on a position-sensing surface. Refers to multiple-value input being either time-multiplexed (one mouse or stylus) or space-multiplexed (two hands).
multi-point touch-sensitive surface (not described: capacitive touchscreen?) used to read fingerprint when user selects an icon on a touchscreen, access rights granted if match on finger print from database of authorized users.
Overview of problems to be addressed in mobile computing (e.g. tablets, smartphone, PDAs). Communications w. intermittent connectivity, file reconciliation/updates, changing IP address. Small display, Presumes switch to touch/pen computing: touchscreen input less accurate than mouse, display obscured by hand/stylus. Power management (clock rate). EO/GO/PenPoint battery to 6 hours (same as laptop), Newton and Zoomer to 40/100 hours life.
Transparent windows in a GUI, using alpha blending of pixel data. Example show menu boxes with buttons overlaid on other windows.
Short study of Windows for Pen Computing handwriting recognition accuracy, compares to Neisser60, cites also LaLomia94.
Remote control of consumer television set by hand gestures: waving, etc. Video hand-tracking, stationary background removed by subtraction.
IDE development environment with object, code browser, links reference to source code. No layout editor for GUI shown.
Delayed translation/Deferred recognition of electronic ink, part of Slate/Bricklin thinking of ink as a data type. Implemented for PenPoint and Pen Windows in PenApps, a Visual-Basic like language. Refers to leaving electronic ink displayed if handwriting not recognized, or handwriting recognition result not found in dictionary. User can also display ink instead of recognized text. Gestures?
System administration manual for Unix. Discusses ACLs access control lists on AIX and HP-UX versions of Unix, with extended permissions, deny as well as grant permissions/rights.
Machine learning / backpropagation neural net work recognizing handwritten characters and symbols (gestures): training-based recognizer. Multi-stage recognizer: "energy" dynamic features (velocity and acceleration), and 3x3 grid patterns.
Guide to PGP public/private key cryptography and PKI.
Tactile Braille display using cells of electrorheological (electrostatic, not magnetic) fluid.
Collection of reference manuals and papers for Garnet IDE (see other refs) from CMU archive web site as a single publication/reference: includes directory listings.
Critique of Pen Windows / W4PC as only encompassing double-click, not drawing of gestures. Gedrics: gesture-driven icon. Gestures (of single stroke, to be fast) are performed on an icon: tab to activate, right-left to open, single-stroke question mark for help menu, closed "X" to delete, press/hold or hold/drag for copy/paste/move-to-parent, etc. circle left and circle right. Implementation on Newton. Does not mention "marking menus".
Video presentation of General magic smart phone / PDA. http://Slate.computer with stylus touchscreen from Sony. Appears to be resistive film tablet. Calendaring, virtual keyboard, electronic ink for notetaking, Personal Link communication network services from AT&T for email, etc.
Video presentation of General magic smart phone / PDA. http://Slate.computer with stylus touchscreen from Sony. Appears to be resistive film tablet. Focus is on Telescript: early messaging/email service?
Telescript: Prototype-based programming language for pen-centric pen computing applications for General magic platform.
GEOS/Geoworks system documentation for pen-computing operating system on Nokia 9000. Includes discussion of electronic ink, handwriting recognition, API for gesture recognition.
GEOS/Geoworks system documentation for pen-computing operating system on Nokia 9000. Class hierarchy and messages.
Visual programming language / GUI for creating programmed multimedia presentations.
Capacitive/electrostatic grid tablet ("dipole electrodes"), finger affects capacitance at intersections. Coarse position is the grid crossing point with the highest value for change of capacitance. Interpolates between two adjacent grid crossing points for higher fine resolution to determine centroid of finger position. Proximity sensing. Says Rympalski measures changes in capacitance as conductive object comes near. Compare with multi-touch?
Speculative (forward-looking) paper on web-browser for PDA Apple Newton: advantage is ubiquity and portability, disadvantage was high wireless data charges while mobile, small display (four-way scrolling particularly difficult) and small memory. Deferred fetching of images (low bandwidth through workstation/PC to dial-up Internet connection). No bibliography: does not mention Lynx browser.
See also 1997 WWWC paper by O'Modrain. Force-feedback stylus, intended as GUI user interface accessibility for the blind/visually impaired: part of virtual musical controller interface (buttons, sliders, etc.). X/Y force actuation (no Z) through X and Y flexures (flexible linkage). Simulated Z force by changing apparent virtual resistance of a check box to a vertical pyramid vs. flat box.
Textbook/reference on 3D graphics algorithms: mostly curved surfaces, hidden surface removal, with emphasis on real-time performance (so nothing on ray tracing or luminosity). No masking.
3D/6DOF user interface for surgeons viewing CAT and other scans during surgery: Polhemus 3D digitizer inside doll's head, surgeon moves doll's head to show projected 3D image. Ignores X/Y position (so really 4DOF?), Z is used only to zoom image.
Tactile feedback mouse with two actuators under mouse buttons, two under fingers for sideways acceleration.
Capacitive grid digitizer, transparent on top of display. Can sense both finger (capacitance), and signal radiated/received by a stylus. Mentions microcracks in ITO / brittle for tension forces, so put on outer side of curved plate: the forces are compressive.
Extensive list of video tapes from ACM CHI, CSCW, and ACM SIGGRAPH Vieo Review relating to handwriting recognition, sketch input, signature verification, and pen-computing: 1983 to 1994.
1963 Video: Videotape of Sketchpad system by Sutherland.
1985 Video: Software Control at the Stroke of a Pen - Pencept - CHI '85.
1989 Video: The Paper-Like Interface, IBM Watson Research Center - CHI '89.
1989 Video: Freestyle - Wang Labs - CHI '89.
1992 Video: Combining Gesture and Direct Manipulation - CMU.
1992 Video: Touchscreen Toggle Design - Univ. of Maryland.
1992 Video: Tactile Manipulation on a Digital desk - Rank Xerox EuroPARC.
Video: Alternate Reality Kit - Xerox PARC.
Video: The Lapidary Graphical User Interface Design Tool - Univ. of Tennessee.
Apple Newton: selected text in text editor can be copied/pasted by drag-and-drop: shaded image of text is what user drags around. Compare with floating icon for drag-and-drop in PenPoint?
pen-based drawing program with glyph recognition (sketch recognition) and spatial constraints. Commands/gesture mode by side-button on stylus: compare with Pencept? (not cited). Multiple recognizers. Contraints defined in separate constraint language. Cites to visual languages, constraint languages.
Capacitive grid touchpanel/touchscreen (transparent) checking capacitance at intersections of two layer grid, finding center of gravity of area/zone/region affected by finger. Used for touch keyboard. More "zones" of detection than lines and rows. Compare with Rekimoto?
Overview of PEM Privacy Enhanced Email PEM standard: certificate based encrypted and originator-authenticated/signed email. File also contains RFCs.
RFC 1421: Privacy Enhancement for Internet Electronic Mail: Part I: Message Encryption and Authentication Procedures (1993).
RFC 1422: Privacy Enhancement for Internet Electronic Mail: Part II: Certificate-Based Key Management (1993).
RFC 1423: Privacy Enhancement for Internet Electronic Mail: Part III: Algorithms, Modes, and Identifiers (1993).
RFC 1424: Privacy Enhancement for Internet Electronic Mail: Part IV: Key Certification and Related Services (1993).
Touchscreen PDA with full keyboard, swivel/convertible/clamshell display. Rotate screen between horizontal for keypad input and vertical orientation for pen input. Graffiti Handwriting recognition software, GEOS operating system. Paper file contains additional materials.
Functional touchscreen PDA with full keyboard, swivel/convertible/clamshell display. Rotate screen between horizontal for keypad input and vertical orientation for pen input. (Screen is square, rotation is immediate.) Pop-up tools have window frame and dragable title bar. Graffiti Handwriting recognition software, GEOS operating system. Paper file contains additional materials.
Touchscreen PDA with full keyboard, swivel/convertible/clamshell display, penGEOS software. Front cover shows OmniGo 100 with both portrait and landscape screen orientation. Trademark on "hotsync" "hot-sync" (?) via OmniGo connectivity pack (Windows software plus serial RS-232C cable). GeoWorks Bindery for electronic books with hyperlinks in text. IZL (Interpreted Zoomer Language) programming environment for GeoWorks. Septor wireless pager/news-alerts services.
Windows software and serial cable for syncing and transferring between OmniGo pen PDA and a Windows PC. Trademark on "hotsync" "hot-sync" (?) via OmniGo connectivity pack (Windows software plus serial RS-232C cable).
Pen GEOS operating system documentation.
Looking for optimal trade-off of transparency between user attention being focused on one of two images, versus divided attention between the two (Stroop Test). Show semi-transparent windows, semi-transparent pull-down menus, pop-up menus over graphics and over text. Also shows 100% transparent, with only menu labels / lines visible (in contrasting puke-green color) with shadowed outline for contrast. See Rockstar Samsung.
See-through/transparent image over background image gives illusion of depth (depth multiplexing): superimposed windows, overlapping windows. Also changing physical depth of images (semi-silvered mirrors) in 3D. See Rockstar Samsung.
Show semi-transparent pop-up menus (including pie menus or context menus) semi-transparent or wire-frame over object, so that user can still see context, underlying image/object not obscured. See-through UI tools. See Rockstar Samsung.
User interface with semi-transparent windows, menus, dialogue boxes, other objects with attention to "layered" interface objects, click-through tools. Cites to Knowlton on dynamically re-labelled buttons and keys. See Rockstar Samsung.
Commercial product that finds literal names based on capitalization, and dictionary, plus knowledge engineering rules. Compare with regex expressions to find phone numbers. Finding names, combine with UI?
Xerox data glyphs, bitmap printing to encode structural and other information in printed documents: links, etc.
Xerox data glyphs, 2-D encoding of information in small pattern within printed images. Diagonal pattern segments, 0 and 1 representations at right angles, but small enough that do not appreciably affect visual appearance. Made robust to rotation by encoding reserved pattern segments that are used to recognized the orientation.
Gesture (?) for turning a page by touching edge of page in a simulated book, and moving animated page(s) to the left or right. Compare with flick gesture, PenPoint OS metaphor of notebook pages?
Translucent/semi-transparent window on OS/2 so as not to obscure image underneath (touchscreen/mouse). Blending for transparency is done by checkerboard of BITBLT, so that background image with colors appears visible.
Hybrid user interface: touchscreen UI can be combined with 3-D input device, because user can reach out to touch touchscreen with the 3-D input device in their hand. Relative vs. Absolute gestures: hard to perform gestures in absolute coordinates. Bi-manual/two-handed input advantageous over sequential single-hand input, because user can (e.g.) position a tool with one hand and use the tool with the other. Mentions using multiple pointing devices in a prototype (trackball and mouse, or two mice). User's spontaneous use two hands for combining translation/rotation/scaling gesture (direct manipulation). Compare with multi-touch.
Scroll to line, paragraph, or other document boundary. Scrolling initiated by minimum gesture movement, with second threshold to stop autoscrolling or scroll back if touch location has not moved on touch-sensitive surface.
Touchpad controller for very small touchpad: uses (optical) sensing of velocity to position cursor faster if touch is moving fast.
Introduction to Smalltalk language and environment. (Electronic file.).
Invariant handwriting features (independent of rotation, translation, scale) are ratio of tangents and normalized curvature, for Hidden Markov Model recognizer.
Stochastic pattern recognizer, Hidden Markov Model (HMM).
URL as of 5/9/2002: http://www.research.avayalabs.com/user/jianhu/index.html with additional information on Jianying Hu.
States that using a sliding window in a recognizer reduces error rate by 50%, but does not give the point of comparison, or the description of how the error was computed (false positive, false negative, etc.).
Resistive-film capacitive touchscreen with front shield to protect from stray capacitance.
Acoustic wave touchscreen: hard stylus presses on top elastomeric layer, which then absorbs acoustic energy of wave propagating in layer underneath.
indexing of electronic ink notes by matching scribble markings: states 97% successfully matching rate (compare with character recognition rates of 97%), Scribbles/blobs taken from fields: e.g. "To:" fields in notes/email. Feature points are knots/inflections and syntactic rules for sequences, compare with chain codes.
Architecture of IBM Pen for OS/@ V1.0 and PenDOS from IBM, version 2.2: Lists gesture assignments by Session Type, Gesture Commands and Sessions.
IBM Simon user manual. Touch screen uses plastic stylus or finger. PredictaKey keyboard shows six most likely characters on virtual keyboard, Other shows next six most likely. User can highlight numbers in messages and they are dialed from a pop-up menu. Physical slide switch p. 4 for On/Off / Resume/Suspend, p. 8 Simon is not completely off. If not off, touching screen turns unit on. Buttons with broken-line edge are not available: drag stylus across screen to highlight button, then lift stylus to select that feature. Some buttons display check mark on/off on touch. Folder-shaped buttons contain additional buttons. Has search function for address book p 35. Specifically states that operations happen "when you release the button" e.g. pen/stylus lift, not on pen/stylus down -- carry-over from mouse development?
Windowing system uses different control layout from Microsoft Windows. Drag-and-drop to printer icons. Style manager allows color palette for all windows to be updated. Three button mouse, pop-up menu button can be rightmost of leftmost button depending on handedness of user.
Unnarrated demonstration of IBM Simon from BellSouth touchscreen smartphone, circa 1994.
Demonstration of IBM Simon touchscreen smartphone, circa 1994, from reporting on CHI 2011 conference.
IBM Simon touchscreen smartphone, circa 1994, product information, press articles, photographs. Separate applications/apps for Simon, zoom camera in 2005.
Pen/OS2 pen computer: every pen touch-down event on an object, object determines how stroke should be interpreted: e.g. mouse-mode in title bar, mouse+gesture mode on button, mouse+gesture+single-character-recognition in list box, etc. User can override definitions manually, or by holding stylus side-button. Compare with "gesture buttons", ink interpretation in PP and PW?
Readme file (and file listing) of installation media for Pen extensions to OS-2 operating system, 1995. "Enable Edit Window" -- first point of contact (in one edit window) causes all other input to be directed to that window regardless of where. "Telepen" whiteboard application? Some incompatibilities with pen as mouse.
Computer hardware technical dictionary.
Overview of problems in handwriting recognition, both OCR and on-line. Includes discussion of loop/cusp transition (i discussion on segmentation of script characters).
CodeWarrior by Metroworks, Object Master, MacAPP, Workshop. Development environment: gives 1994 or earlier date for MacAPP and CodeWarrior.
Microsoft WinPad palm-top/hand-held pen computer, high price and release delays resulting in Compaq not releasing Mobile Companions. (WinPad later named Microsoft At Work for Handhelds.).
InkWare NoteTaker note-taking application for pen computing: electronic ink plus SmartStamps (selectable tools) classifying notes as ToDo, Urgent, Call to appear in automatically-generated index. Also attache pre-defined keywords to ink annotations. Ink-as-datatype (compare Slate PenScheduler?).
Appears to have no relation to Inforite82 product, separate company? Several companies have used the name "Inforite"?
Intel386 embedded processor, includes registers and MTL6560 digitizer controller for resistive touchscreen (digital voltage measurement), controller/co-processor for LCD.
resistive touchpad using two sheets separated by an air gap, four-wire sensor. Based on contact resistance, also determines touch pressure (force). Tap-and-drag gestures. Drag Edge lock: if drag to edge, maintains drag gesture status for three seconds after release so that drag can be continued/extended with an additional drag/wipe gesture. Further gestures include right-click by tapping in upper right corner region of touchpad digitizer, right edged border scrolling for up/down scrolling.
States large hardware business for specialize electronic chips for PDAs with electronic ink, handwriting recognition coprocessor. Mentions immaturity (performance defects) of digitizer tablets. e-ink.
User interface, set windows to transparent so that user can see the information they have selected.
Handwriting recognition UI characters translated to keyboard input: no references to Sklarew/Linus or Ward/Pencept?
TeamWorkStation-1/ClearBoard: Whiteboard systems: Combined videoconference image in shared drawing space on workstation display. Using transparent display, other user appears in video on through transparent display, writing reversed. Cites to Media Space shared drawing space from 1988. "Transparent digitizer sheets" -- does not say what touchscreen technology used.
Describes mouse as absolute position device if not lifted from pad (sic), thus a three-dimensional digitizer is equivalent for two dimensions. Refers to device-independent API interfaces such as ACM CGS. But in practice, practical attributes of input devices are not separable from types of user interfaces and interactions.
Wearable computer, a PC (with standard PCMCIA cards, etc.) with components distributed in pockets in a "utility belt", cables/connectors for system bus etc. running through belt (or suspenders). Refers to foldable/portable input/output device for pen-based LCD (or keyboard/touchscreen).
LabView graphical programming language for software simulation and control of virtual laboratory instruments, describing dataflow programming. Text refers to RUN switch in examples (p. 84..92). Introduction gives history of LabVIEW to 1986. User interface: notes that scroll bar on MacIntosh and Windows effectively halts asynchronous/multi-tasking operation.
GUI ToolGlass two-handed/bimanual user interface: non-dominant hand positions an object or tool, dominant hand uses or manipulates the object or tool.
Area cursors for accuracy in pointing on touchscreen/tablet for 2D and 3D pointing tasks. Moving/dragging/scrolling file into a folder; Toolglass and magic lenses; Silk cursor with homing grid. "Prince" technique named after brand of oversized tennis rackets.
Overview of tactile/haptic displays 1995. Force-feedback joysticks, Optacon, pressure sensors on forehead for insensate fingers, tactile vision substitution, sign-language and fingertip lip reading (Ladoma) for deaf-blind. Spatial resolution of human fingertip sensing.
Survey of mathematical expression editing, mostly from a keyboard (and mouse?). Section 4.10 talks about pen computing user interfaces: two-dimensional input is good, but large number of symbols used is problematic for handwriting recognition.
User interface with touchscreen, electronic ink, gesture commands, and handwriting recognitions. System determines how to interpret user input based on context such as size relative to known characters, location of input on/over/away-from existing ink or characters, whether the input document already exists, and relative priority of gestures.
Additional materials in file: http://winster.com, etc. historical notes on PenPoint, concerning Jerry Kaplan.
History of GO Corporation and PenPoint, experience with IBM and Microsoft, from point of view of CEO of GO. Compare to account of same events by Marlin Eller in "Barbarians led by Gates" about PenWindows.
Additional materials in file: http://winster.com, etc. historical notes on PenPoint.
Study of testing handwriting recognition algorithms using some 87,000 characters from 150 writers. Supported by Microsoft Pen Computing Group: Greg Slyngstad and Sunh Rhee. (Cites variability models of Ward and Kuklinski, but does not derive implications about 97,000 being a small sample size.).
Haptic mouse with force feedback, integrated with X windows. Electromagnetic actuator in mouse, feedback provided external to host CPU because control loop through host application too slow or too low in bandwidth. Turns mouse into absolute position device?
Proposed API for handwriting recognition systems, using pen strokes optimized for point vectors in the X window system, but otherwise independent of OS or windowing system.
Incremental translation/recognition of handwriting.
support for context as for Japanese and Cursive English.
API for character prototype files and dictionaries.
Data structures for gestures and arbitrary objects.
Data structures correlations between recognized objects and pen strokes.
Handwriting recognition system for pen-based computing for Solaris: cited to PenWindows/Microsoft Windows for Pen Computing (MWPC), PenPoint, Apple Newton for pen-computing. Solaris API based on PenWindows API and Penpoint. Not integrated tightly with windowing system: multiple window toolkits on Solaris (XView, OLIT, Motif, COSE, Interviews). Stateful implementation: can acquire context information from previous strokes. Recognition subsets of gestures, mathset / math symbols, Kanji (JIS1 subset only), Katakana, hiragana, etc. Support for delayed strokes vi correlation.
Tactile feedback/display on mouse by varying resistance/force on mouse button. Used to give apparent "height" to menu items similar to enabled/disabled grayed-out display, variable resistance/force to distinguish multiple buttons, etc. Proposes use for accessibility of GUI for the blind/visually-impaired. Cites to earlier work on tactile vision substitution.
Tactile feedback to improve accuracy of target pointing/selection on touchscreen. Attractive force when close to target, repelling force when not near. Cites to tactile / force-feedback mouse.
Averaging out the prototypes in adaptive handwriting recognition system.
Transparent overlay windows in a user interface.
Third volume in "Graphic Gems", collection of papers: Image Processing (bitmap compositing), Numerical and Programming Techniques, Modeling and Transformations, 2D Geometry and Algorithms, 3D Geometry and Algorithms, Ray Tracing and Radiosity, Rendering.
On-line help, with hypertext linking and search for related topics by text.
Capacitive grid/matrix of conductors used for touchpad sensor for fingerprint scanning. Compare to grid arrangement of Rekimoto.
OS for hand-held personal information manager: Telescript language for application implementation. Touch-sensitive touchscreen, stylus or finger. Refers to targeting "halo" for objects. (Compare with Slate PenApps "gravity"). Tap/Touch gesture terms used interchangeably. Desktop/Office/filing-cabinet/hallway metaphor. Electronic ink note-taking. On-screen keyboards: brings up "smart guess" keyboard (numeric, etc.) based on syntax of input.
Simulation of indexed flip book / rolodex in Wang Freestyle, with small edges of pages shown.
Feature in LabView visual programming language for virtual laboratory instruments. Fig. 5, text page 70 refers to slide switch: compare to Apple slide-to-unlock?
Hand-held touchscreen terminal with bar code / optical scanner, handgun-style grip. Compare with Termiflex?
History and technical description of Kerberos authentication system through version 5, at MIT Project Athena. Single sign-on: user login gives ticket used by all application. Uses secret-key (single key) cryptography, version 5 addressing interoperability with public-key cryptography: cites to RSA digital signatures. See also earlier 1991 publication.
Vibration haptic feedback on remote-manipulation display, combined with/without force feedback. Illustration is two finger "waldos" to prevent user pressing too hard.
Overlapping translucent windows (or non-rectangular patches), so that users can annotate on top of other windows. Implementations: re-drawing (non-pixmap?) on top of other drawings after lightening lower window, changing color of lower pixels by translucency of pixel above (alpha blending). Interesting approach on sketching, electronic ink. Patch-related drawing gestures: line into (pearl), zig-zag clear (erase), zig-zag over border (cross-out/delete), small circle for pop-up, clockwise/ccw spirals. Gestures vs. ink by side-button on stylus/pen tablet (compare Pencept recognition macros). Overlapping windows may cause lower patches to render differently (e.g. not show control elements, etc.) (cites Toolglass). See IntVen Motorola, Rockstar Samsung.
User interface to select objects using selection rectangle (objects may intersect edges), the selected a tool: Scale, Skew, Rotate, etc. Moving rotate tool changes/sets pivot/rotation point.
Low-latency VR virtual reality display of simulated patient anatomy, Polhemus sensors on shutter glasses to determine viewpoint of user's head, latency somewhat less critical on hand gestures.
VR virtual reality display system, user task areas are physicians, medical training, automotive engineers, architects. Voice / speech recognition command UI (e.g. zoom in, rotate), head-mounted Polhemus sensors for calculating positions of each eye for stereoscopic display. DataGlove for manipulating virtual objects (e.g. bones in anatomy).
Dual-display device, two displays can be used as one (two pages of a book), or used separately. Compare with devices like Refalo? Uses term "electronic pen" for touchscreen. May be used as add-on display to desktop computer. Thumbnails of document pages on one display, full page on other. Can display virtual keyboard on one display or portion.
Use of pop-up radial menus to train users to "gestures" which match the motion used to select from a radial menu. Also display of images of gestures in menus, and pop-up text giving name/meaning of gesture recognized. Circle/lasso gesture around an object brings up a pop-up menu of actions that may be applied to the object. Also shows pig-tail gesture for delete/erase, scroll-up and scroll-down (flick?), lasso gesture for select. Gestures may be recognized using velocity: e.g. Slow flick moves to next page, fast flick to end of document. press-and-hold/"pressing and waiting".
Marking menus on MacIntosh/Apple. (pop-up pie menus when user hold stylus still, selectable without first lifting pen). Zig-zag gestures for navigation. press-and-hold / press-and-wait for 0.3 second.
Pressure sensitive tablet digitizer using two conductive sheets and conductive rubber.
Cited along with Neisser (93% recognition success for humans without context), that users not satisfied with 97% correct character-to-character handwriting recognition success.
pop-up virtual keyboard in GridPad computer, in a particular form application. Compare to Soft input panel SIP?
Basically a PDA/smartphone/always-with-you-computer with notes, reminders, calendar, video diary that is also sensitive to user's context (especially location). Intended to help look up information, or recall information for user. Cites to Memex as collection of user's notes and documents. Memory aids: e.g. easier to remember who was at meeting if other aspects of meeting are recreated / reminded. Cites to Amstrad PenPad PDA, EO 440 Penpoint, Newton Messagepad, etc. Main example is Xerox PARCtab, touchscreen, wireless/infrared, active badge (fine-grained location).
SILK: User interface prototyping via storyboard: user sketches dialog or UI image with electronic ink on tablet, specifies user action (e.g. button press), sketch new UI appearance. Wacom tablet on Apple Macintosh: Implemented with Garnet (see reference).
Front-mounted electromagnetic digitizer (CalComp) with a display: gives long description of Scriptel front-mounted electrostatic display, discussion of problems of rear mounting. Uses term "pen-driven computing". Two layers of grid are fabricated on two layers of transparent mylar.
Forms-based inputting gestures to control recognition, insert, delete scroll -- compare with GO? Refers to smoothing (and averaging?) to obtain the non-distortion point on stylus input filtering? Says pressure-sensitive touchpanel over display (touchscreen). Cites to Sklarew. Handwriting-box input area -- compare with Pencept? Mentions correction for offset/discrepancies between touchpanel and display.
On-screen toolbar for selection one of several on-screen virtual keyboards or handwriting input, toolbar can be disabled: handwriting recognition shows multiple candidate results, user picks correct one. Toolbar can be repositioned on screen: compare with Pen Windows and PenPoint?
history of handwriting recognition (?) back to 1950's.
Study of digraph/trigraph dictionaries in virtual keyboard typing correction: a single trigraph dictionary not as useful as multiple digraph/trigraph dictionaries with different word populations / vocabulary for appropriate domains.
Lexicus neural network handwriting cursive recognition: 25,000-word dictionary and statistical methods (compare with Viterbi algorithm?).
When zooming on graphical image/map, show birds-eye or world or greater context view as translucent overlay on window, so that user can maintain visual context. Zooming on file system hierarchy zooms into files, without explicit opening of file. Cites to Bier for "see-through" interface (transparent pixels instead of translucent overlay).
Microtouch touchpad digitizer: press-and-drag distance is a minimum distance threshold before user input does drag automatically by simulating mouse button action. Dwell/press-and-hold for minimum distance threshold. Motion can coast/decay after user lifts finger (e.g. inertia plus friction).
GPS aid for visual impaired/blind, giving information about surrounding environment as well as location. Output to user by means of artificial auditory beacons located at correct/appropriate binaural location relative to user.
Avoid need for handwriting recognition when indexing documents by approximate matching of hand-drawn pictograms. Mentions difficulty of handwriting recognition of (Asian) languages. Compare with Slate DayTimer/PenScheduler for ink indexing, Windows for Pens NoteTaker.
Searchability of electronic ink; approximate matching of hand-drawn pictograms, not just comparing with character recognition results to index handwritten text.
Review of pen-based computer (PenPoint) with applications for engineering: notetaking, engineering design. Applications written in Slate PenApps. Generally, paper fared better than electronic notebook, in part because notebook was not perfect writing surface and was a bit small (6" by 9"). Notes that engineers dislike doing documentation, in part because words are not the natural medium of expression -- diagrams/sketches are preferred. Cites several other electronic/virtual notebook systems, but most not portable.
Project planning/project management application. Tasks, Milestones, assignment of resources and personnel, project tracking. Supports critical path method CPM. Views include PERT charts, Gantt charts, Detail charts, Calendar views.
Computational rendering and display of holographic (stereo vision: full 3D possible) images in "patient" real-time (2 seconds). Can also render a real-world scene captured from a linear array of cameras.
Pen computer can be rotated so that screen orientation is portrait or landscape, for right-handed or left-handed user. Mechanical button to select orientation.
Determine user intent by applying a pattern template to user interactions in a pen user interface, and then carry out an operation automatically.
Also check Capps file.
Speed of entering text on pen-based (PDA/tablet) computer: on-screen/soft keyboard with QWERTY basest, then hand-printing recognition, on-screen/soft keyboard in two-row Alphabetic arrangement slowest.
Application of Fitts' law to 2D and 3D user interfaces.
Product announcements on Tarpon, ruggedized Newton MessagePad with wireless modem (Ardis network), GPS, cellular phone ("external industrial phone receiver").
Apple MacIntosh trackpad: capacitive grid, able to detect size of finger contact (pressure-sensitive). Tap gesture not used, separate thumb buttons.
Description of Wellner-DigitalDesk-like interactive project for capturing handwritten (paper) updates to CAD drawing. Overhead Projection of virtual paper onto physical paper, location of physical paper tracked optically. Live video communication between users.
Shows pen/stylus based PDA with office desk / office room and hallway metaphor. Appears to be a resistive touchscreen, looks like it might be a Data Rover 840.
LCD repetitive row-inversion signal interference with touchscreen. Disable touchscreen during spike interval. LCD functions as array of capacitors, spike coupled both capacitively and inductively. Capacitive touchscreen using sheet.
Dynamic time warping / Time Delay Neural Network for single-character recognition, using a 3x3 or 20x30 bitmap local image.
Claims 99.5% recognition writer-dependent, between 91.5% and 99.5% on writer-independent: expansion to larger writer-independent database gives about 75%.
GUI frameworks comparisons: Macintosh Desktop, OSF/Motif Desktop (with three-button mouse operations BSelect, BTransfer, BMenu right button), NeXTSTEP Desktop, IBM CUA Workplace Shell, Microsoft Windows Desktop Environment. Does not cover PenPoint, page 191-ff covers Microsoft Pen Windows extensions: press-hold-drag is a selection operation, PenWindows 1.0 gestures Extende Backspace, etc. Direct manipulation. Pop-up menus on page 34 and 202. Describes drag-and-drop and pop-up menus as OOUI, object-oriented user interface, not GUI.
Projection whiteboard system with LCD project panel, overhead projector, digitizer. See other references to MartinD.
Describes alignment/calibration method for display and projection vs. tablet, similar to that of PenCept/CIC/etc. for paper versus tablet.
Knight-Ridder Information Design Lab: electronic newspaper on a pen computer, with pen/touch user interface.
Bibliography of papers relating to document understanding, starting about 1980. Sections include: On-line character recognition; Gestures and Sketches; Script; Foreign optical character recognition; Recognition of mathematics and Formulas; Line drawings; Signature verification; Writer Identification; Synthetic data; Forms processing; Shorthand; Music recognition.
Combination 3D pantographic digitizer input device and force feedback output for haptic feedback. Very low mass, low inertia, mechanically balanced. For fingertip of one hand: feedback is at single point in user's fingertip. Virtual buttons give feeling of physical clicking.
scrollbar slider: user can control speed of scrolling (for finger control) based on pulling a virtual elastic line farther or less (or changing direction of pull). Cites to AlphaSlider. Note: cited in at least one scroll/bounce patent by Apple.
Chapter 5: Interactive learning / Concept learning system Cima, Turvy, and Moctec. Example is teaching system to find phone numbers in notes, in different contexts and in various canonical forms, and second component that learn so interact with users and application programs, (dialing numbers). See also "Instructable agents" (spelling error in dissertation title).
Slide an icon onto a position to unlock: slide to unlock?
Pie-pad pop-up menu for numeric input, pop-up circular menu with digits. Marking menus. Mentions Goldberg 93 paper on unistroke.
Power saving by waking up device (tablet) when user touches or brings stylus/finger near tablet. Timeout to go to low-power mode.
Includes reviews of several experimental systems of the early 1990's, and a lengthy bibliography: several German references included. Almost all the references are from 1991 to 1995, very little before that.
Unistrokes. Jotted notes: implying electronic ink as a data type. PenPoint, Windows for Pen Computing. Cites Alan Kay for early vision of handwriting computer (although no such device was ever built?). Cites early paper by Ward, but not the on-line bibliography (1995). Rotatable orientation of pen computers: landscape/portrait.
touchscreen/tablet/digitizer: mentions dual sensing of stylus/pen or finger or both, with positions of each.
Tap and hold (TapNHold) gesture: tap analogous to clicking left mouse button: TapNHold click tip of pen on tablet and holding for a specified amount of time. German edition of (discontinued) support article documents Windows for Pen Computing 1.0, prior to Windows95. Tap/press-and-hold on Pen Windows.
Windows for Pen Computing as portable information at your fingertips: reference "vision" video of Information at your Fingertip in 2005, with face-to-face video in mobile settings, pen computing, video conferencing.
Windows for Pen Computing version 1.0 user manual. Pen tools group includes Pen Pallete (soft input panel for writing/keyboard), on-screen virtual keyboard. Sample form has boxed handwriting input, unboxed input with timeout for word segmentation.. Checkmark gesture to edit text (similar to PenPoint), pigtail gesture to delete, space-insert gesture with hotpoint. Tap and hold gesture (press-and-hold) to start selection stroke. Newline, copy (sideways pigtail), paste (inverted V), delete, delete word (right-left-combined), extend, tab gestures.
Cursor automatically turns to pen for handwriting input in application when cursor is in a text-input field. User-defined gestures are circled letter gestures. Trainable recognition (note that Toshiba unit ships with CIC recognition). Control panel settings for double-tap gesture time and area settings, press-and-hold delay. Press-and-hold is effectively a separate gesture recognized by distance threshold and time threshold: moving before time threshold starts a gesture or ink (with recognition).
Handedness setting changes location of menus. Control panel rotate setting to switch orientation of display landscape/portrait. Notebook application: barrel button on pen to display ink eraser; deferred recognition: select ink (by tapping) then select text tool. Recognized text optionally sized according to ink dimensions. Shape recognition (circles, squares). Equations (= at end) automatically calculated; outline indexing of notes. Chart of recommended printing styles.
List of third-party accessibility extensions for Windows and Windows NT. Screen reader / screen review, screen enlarger, on-screen keyboard / virtual keyboard. Vendors include Magic Deluxe / Microsystems Software, MAGNUM Deluxe / Arctic Technologies, MagniPORT / Microsystems Software, Number Nine (hardware pan and zoom in display card), Vista / Telesensory Inc., Zoom Text / Ai Squared, JAWS, outSPOKEN . On-screen keyboards include word prediction, click or dwell mode / AutoClick. Screen Keyboard (SKB) in Windows for Pen Computing.
Microsoft Pre-release feature description for Chicago / Windows95. Plug-and-play, Toolbar (for cut-and-paste, etc.) in MS-DOS command window. Support for multiple pointing devices (mostly for accessibility: alternative head mouse, etc.). Nothing on SIP Soft Input Panel, PenWindows or tablets.
Windows UI guidelines include keyboard, mouse, and Windows for Pen Computing / PenWindows gestures: General circle-around-UC-letter gestures, has circle-M/M-circle pop-up "context menu gesture" / "pen menu gesture". Fundamental behaviors defined for a Pen are Pressing/Tapping/Double-tapping/Dragging. Gestures leave ink. Lasso-tap (circle-tap) selection gesture. Tap on text box brings up text-input handwriting window, with alternative word list or alternative character list from recognizer.
Partial print copy in physical file -- see electronic copy at http://www.ics.uci.edu/~kobsa/courses/ICS104/course-notes/Microsoft_WindowsGuidelines.pdf (fetched 2010).
stylus or finger, pen can perform equivalent operations of mouse.
Defect in tablet/pen driver in Windows for Pen Computing: circle-m context menu gesture does not emulate right mouse button, other problems with barrel button on stylus.
Similar topic to Microsoft95a -- includes section on special design considerations of Accessibility. Some sections on pen input. Guidelines on targeting of electronic ink to fields (but not built-in? Compare with Slate PenApps "gravity"). Lasso-tap gesture to select. Need larger tapping (double-tapping) target with stylus than with mouse, similar considerations for targeting of lasso-tap selection gesture on text. Pen menu (context menu) gesture is circle-m. Describes activate-on-lift of mouse being applied for pen: "Do not activate the window unless the user releases the mouse button (the pen tip is lifted) in that window." Press-hold-drag for select instead of barrel button. Auto-scrolling based on velocity. Mentions accessibility "Screen Review Utilities" (screen readers) for the bling, on-screen keyboards. Pop-up/context menus synonymous: refers to contextual interfaces such as pop-up menus, property sheets, and OLE drag and drop. pop-up (context) menu handlers. Press-and-hold for select, p. 55.
Partial print copy in physical file -- electronic image file.
#include definitions file for Windows for Pen Computing Version 1.0 -- 1992? Pen tablet data OEMPENINFO include pressure (force), height (proximity), angle X/Y and Z, barrel rotation. Standard gestures for clear, extendSelection, undo, copy, cut, paste, clearword, kkconfer; AnsiGestures (ASCII?) backspace, tab, return, space; application gestures circle a-Z and circle A-Z; shapes line, ellipse, rectangle, min and max.
See also penwin.h for version 2.0.
Accessibility on-screen visual/virtual keyboard systems for accessibility: compares to single-switch Morse code. Eek!; Microssytems Software HandiKey; My-T-Mouse; SAW Switch Access to Windows; SoftType; WiViK2 Windows Visual Keyboard;.
Features include word prediction, dwell-mode pointing, access grids, mouse emulators (head-mouse, etc.), keyboard macros.
See also similar Microsoft technical notes from other years.
Device to convert absolute co-ordinates of touchscreen to mouse relative-motion coordinates, and having "buttons" recognized on the touchpad for the mouse button: prior art include MicroTouch Unmouse cited 1990. Compare with Mouse Mode on 1980's / delta mode on 1980's tablets?
Compare to Schumer patent on tablet controller, where driver functions are claimed if located in an external controller for a tablet (or mouse?).
Sketch/diagram editor employing constraints, and dealing with inconsistent state during direct manipulation editing: flowchart, hypergraphs.
Force-feedback joystick, with multiple force actuators. Software to create feel of textures, virtual spring and inverse spring forces, etc. Includes reference manual for Sandpaper software.
When selecting objects by selecting a range (drag selection rectangle?), take into account time ordering of when objects were created: user is likely to want to select objects created by the user at about the same time. Uses a white board with rear projection.
Whiteboard system (Xerox Live-Board Tivoli / MeetingBoard on Windows PC) with free-form interaction -- primarily electronic ink scribbling: recognized rough layout of lists, text, tables, and indented outlines in electronic ink, based on informal structure or "visual grammars". Mentions aha! InkWriter which formats/indexes ink based on handwriting recognition, but does not compare with van Raamsdonk which edits directly electronic ink. Selection via various bracket and corner gestures, insert via caret gesture. Has "table clean-up" function to align slanted writing lines of electronic ink.
Handwriting recognition on pen computer of mathematical expression input in a two-dimensional a calculator. Compare with PenPoint math paper, work at MIT 1960s.
Port to Windows of Unix Programmer's Workbench: compare with cygwin.
Port to Windows of Unix Programmer's Workbench: compare with cygwin.
Simplified unistroke-like gestures, as substitute for writing full characters. User writes on a 12-segment outline of characters, outline in shape of figure eight, similar to seven-segment display. Punctuation and some other characters input via pop-up keyboard. For Apple Newton. (See also AlphaPad00).
Radius Inc. Display: rotating display between landscape and portrait orientation results in all windows being rotated, windows may also automatically layout GUI elements.
Motif Window Manager: 1994 version, updated as of 2001. resource XmNsnapBackMultiple sets snap-back/bounce distance if use moves mouse cursor too far outside of bounds of scrollbar widget. Automatic Layout of GUI elements? XmNselectThrehold: click and drag minimum distance that pointer must move to select text on a drag.
Motorola Envoy communicator: PDA with General Magic Magic Cap, stylus/finger touchscreen, fax, wireless (ARDIS cellular radio) /wired modem. PenWare PenCell spreadsheet, AOL-online, Pocket Quicken, online maps, terminal emulation (e.g. vt100), on-screen virtual keyboard, electronic ink. No voice phone.
Video demonstration of IBM Simon touchscreen smartphone: games, apps, scrabble, text messages, data plan. Describes form factor as "brick".
in Garnet system papers: Prototyping user interfaces can be difficult, because test-subject users may not behave like real users, may even have a hidden agenda. Coupling of user-interface to application (via application call-backs) is a nightmare in maintainability. Shows dynamic menus.
Garnet system: Constraint toolkit for UIMS, including object management in user interface.
Garnet system: Constraint toolkit for UIMS, including object management in user interface.
Garnet system: Direct manipulation user interface and IDE with Interactor objects for mouse, rotating, text editing, gesture recognition. Include Lapidary to create widgets or objects, Marquise tool to demonstrate/animate user interface behavior of applications.
Copy of presentation video in electronic files.
Garnet system: supersedes CMU-CS-92-114. States gesture UIs used since 1970's in commercial CAD tools (probably Applicon).
Overview to Garnet System 3.0, gesture-based/direct-manipulation user-interface project. Introduction to full set of Garnet documentation, "The Garnet Reference Manual: Revised for Version 3.0". Extensive bibliography. Garnet: "Generating An Amalgam of Real-time, Novel Editors and Toolkits". IDE which allowed developers to "draw" the user interface using graphical objects: user-interface styles could be applied to all user-interface/objects (look-and-feel independent). Slots (methods): compare with PenBasic from Slate. "X" gesture to delete (using tablet?).
in Garnet system papers: describes X-Windows lack of agreement on widgets, therefore Xt intrinsics so support. 3-D GUI toolkits: Brown Animation Generation System, Silicon Graphic's Inventor toolkit.
See also Myers 93 reference, same title.
Review of Mercator, GUIB: two approaches for tactile/audible feedback for accessibility without a display (telephone menus, blind/visually impaired, etc.). Tree hierarchical menus via keyboard arrow keys with audible feedback on actions (Yes/No, etc.), braille display with touchscreen. Text attributes in braille by underlining/bold/font, verbalization of mouse cursor state (compare with JAWS: not cited.).
FingerMouse: Hands-free/Glove-free pointing input: downward-looking cameras (stereo) recognizes pointing finger when held above keyboard. Shift key used for mouse click (one-button mouse). Concept originated with Francis K. H. Quek.
Practical guide to Security features and practices in POSIX (Unix): MAC Mandatory Access Control, DAC Discretionary Access Control, ACLs, Information labeling. Public key cryptography PKI. SQL and network communications security (e.g. weakness of TFTP trivial file transfer protocol), X.400/X.500 standards, APIs for certificate storage and access (compare CryptoAPI, CryptoKI), certificate revocation lists. Mentions ECMA standard 138, "Security in Open System" with attribute certificates. No list of references? (e.g. Multics).
Optical pen stylus digitizer that takes CCD images of physical ink trail of pen near tip, whole image and writing dynamics can be reconstructed from the sequences of images. 80% (40/50) success rate on reconstructing whole images. Difficulty reconstructing long straight lines. Stress sensor (force sensor) for detecting when writing. Sampling/data rate 10 snapshots/second. Compare with Anoto.
Transparent X/Y digitizer/touchscreen using transparent electrode wires of ITO indium tin oxide. Stylus/pen uses a plastic film coating to get appropriate writing friction on a metal core stylus.
LabVIEW Graphical programming language and software for Windows 3.1: Date approximate. Contains Wells book.
Part of NeXT Computer SDK. Overview of GUI tool for building/making applications. There may be an earlier version: this appears to be 3.X circa 1995. refers to bundles (which can be applications or collections of resources), and projects contained within other projects.
Part of NeXT Computer SDK. Overview of GUI tool for building GUI user interfaces. There may be an earlier version: this appears to be 3.X circa 1995. See also "Working with Interface Builder": build UI from palette of icons/views/components.
Tactile pressure/force sensing skin on cylindrical fingers, and spacing and depth of sensors to "feel" shape of objects with fingers: multi-touch. Strain values used to determine radius of curvature of indentations in elastomeric/rubber skin. Compare with MIT "Teddy"? Review of capacitive, conductive rubber, piezoresistive, electrorheological, and ultrasonic skin sensors that can be constructed in arrays. Design uses capacitive sensors measuring changes in plate separation.
Electrostatic/mutual capacitance imaging proximity sensor for objects: single pair of electrodes to sense distance, pair of electrodes moved over object in a known plane (or otherwise).
IBM Simon characterized as a combination PDA and phone: asserts that voice capability not available on PDAs, though wireless data is. Refers to Simon form factor as a "brick". Compare with PenPoint? Backlit touchscreen (resistive? finger or stylus). Analog data transmission over digital cellular network (e.g. fax), high drop rate.
History and screenshots from WinPAD Beta, precursor to Windows CE. Stylus on handheld computer, electronic ink annotation. Manual change of orientation/rotation.
Includes device for obtaining coincidence between recognition character candidates and character patterns stored in dictionary memory.
General information on PC DOS and PenDOS: mentions "mouse emulation" in PenDOS, separate from tablet/stylus driver: limited use of Gestures in pen-aware Mouse applications. (Web site partially in Polish).
Pen Computer for Windows for Pen Computing. Special VGA display drivers (for inking functionality?) for screen rotation at 0,90,180,270 degrees orientation.
Press report on trade-secret theft lawsuit between VMI and Autodesk, concerning Otto Berkes, later director of UMPC/Origami project at Microsoft.
Review of Mosaic Input, Zone-based handwriting recognition using figure-eight overlay. Awkward to learn, vendor claims 40 words per minute input rate with practice.
Pen tablet PC using Wacom digitizer. Mentions "Dynapad" by Alan Kay: Mistaken reference to Dynabook? Provided with choice of PenPoint, PenDOS, Pen for OS/2 Warp, PenRight!, or Windows for Pen Computing 1.0. Uses ThinkWrite handwriting recognition instead of Microsoft MARS recognizer. Windows has option in Control Panel to set screen orientation, but off-screen controls more suitable to portrait mode orientation.
Casio PDA: does not mention pen or handwriting input. "handy terminal" vendor description. Comparison with Panasonic JT-900PT. Includes printer.
Windows for Pen Computing version 1.0 not easily upgradeable to Windows 95 -- now refereed to as pen service Windows for Pen. Windows for Pen Computing development environment: Visual Basic with version 2.0 pen services penctrl.vbx updated to pen2ctrl.vbx.
Mentions HEDIT and BEDIT (boxed edit) handwriting input controls shipped with Visual Basic. Windows for Pen Computing development environment: Visual Basic with version VB3.0/Pro with provided HEDIT and BEDIT controls.
PenRight by Philip Shoemaker: MSDOS, MSDOS DPMI, and MSWindows pen-enabled platforms. Includes cross-platform support for Graffiti. Windows for Pen Computing development environment: PenRight! for Windows with integrated graphical environment, update to earlier PenRight! Pro SDK. Cross-platform to GEOS (MSDOS based), Casio Z-7000 and Tandy Zoomer PDAs. States that Palm Computing has a handwriting recognizer for MS Windows (Graffiti), but not shipping.
HOPE! for porting applications from Visual Basic to a PDA that does not run Windows: emulated Windows user interface. Pen Computing development environment: supports Casio Z-7000 PDA and Casio PA100 pen computing products, which run GEOS OS.
Description of Newton OS NOS ("Dante") 2.0. Mentions selection and dragging of one icon, or of multiple icons. Feature also in Windows desktop?
Windows for Pen Computing with integrated wireless radio. Handwriting recognition by ART, or cursive recognition by Lexicus.
1994 Video: Demonstration of PenPoint / EO unit. www.YouTube.com comments mention direct manipulation and gesture UI, also demonstration of convertible tablet computer before Microsoft did a convertible. Shows landscape mode versus portrait orientation/rotation. aha! InkWriter shown for note-taking. MagicScript programming. First few seconds are part of a CIC PenDOS / Windows demonstration.
Windows for Pen Computing version 1.0 versus version 2.0, boxed edit control, hedit handwriting edits.
Lisa Sampfli, PenFact Inc. in Boston Massachusetts.
Gestures p. 61: Circle gestures with letter: Lasso: p. 481 shows list of circle gestures by shape, using mnemonic characters -- compare to "recognition macros" of PenCept; HPENDATA object holds chunks of electronic ink data, separate from mouse messages. WM_LBUTTONDOWN / IsPenEvent() / DoDefaultPenInput: electronic ink data is associated as extra data (HPENINFO) attached to a mouse event. Defines stroke pen data between two pen tip transitions, hence touching strokes and in-air/proximity strokes. (What about out of proximity?) Context menu in bedit (boxed handwriting edit) controls. Single tap and double tap on character bring up a (context) menu/list of alternative characters/words. Page 41 portrait/landscape rotation of orientation mode.
Tactile output of GUI for the blind: two-line Braille Display can be moved around screen image in discrete steps (no mouse), image on screen translated to Ascii/Braille characters: brackets, etc. Routing button "drops" mouse cursor at current location of Braille display in window.
User-interface for large surfaces needs UI without static-placed element (e.g. pull-down menus). Pop-up pie menus, hierarchical menus, pop-up handwriting recognition pads, gestures, vocal user interface. Problems studied in example of Xerox Liveobard whiteboard system. Cites pop-up menus to movable menus of Wiseman, Xerox Alto 1976. PenPoint, GO, PenWindows, Momenta.
Graphical/virtual/synthetic keyboard on-screen keyboard on Apple MacIntosh, usable with mouse without physical keyboard. Also includes predictive type-ahead. Main use is as accessibility technology, combined with dwell-mouse for handicapped use.
Wacom tablet integrated into MacIntosh (?). Indexing of electronic ink notes (automatic concordance) by ink matching of stroke groups separated by distance, then matched by dynamic time warping. Both text (handwriting) and special symbols (weather maps) -- no reference to indexing of sketch input. Used as indexing/annotation method in Marquee video logging.
Industry-standard driver from www.pointing.com/LCS-Telegraphics to allow all (DOS) applications to use pointing devices (mouse/tablet/touchscreen/joystick/etc.) interchangeably. Wintab/Windows/Pen Windows: mentions tablet sharing, multiple application accessing tablet at once with different services: different tablet context objects for different (concurrent) applications, tablet packets with data for a high report rate. Driver supports command-input area, WT_PACKET window messages.
Wintab also mentioned in Windows95/W4PC/TabletPC documents.
Also exists in version from about 2003 with minor updates from Wacom, later version 2010 (see file).
DataPen hand-held text scanner. Functional, in original packaging. Compare with Anoto pen.
Cited in Arai 1997 paper for hand-held optical digitizer: appears to be a text scanner. No copy, file is alternative references.
Hand-held computer -- format is like hand-held terminal, running Windows CE, touchscreen used as mouse replacement.
Optical image detection of finger pointing instead of mouse, using color mapping for detection. Mouse click by tapping shift key on the keyboard. Pointing mode for finger tracking by recognizing "pointing gesture" of hand.
Gyroscopic mouse/joystick as 3D input device, yaw/pitch/roll used to control X/Y/X (absolute?) coordinates. Does not require desk space of a mouse. Compare with 3D mouse?
Technology review of touchscreens: flexible resistive film (a la Elographics), Capacitive, Capacitive zone, SAW surface acoustic wave or SAW guided acoustic wave, IR infra-red touchscreens, strain gauge. Refers to problems with capacitive touchscreens. Mentions that acoustic SAW screens can sense multi-touch more than one point of contact. Capacitive proximity sensing (Microtouch zones). Does not use the word tablet. Cites Scriptel/Symbios electrostatic tablet with proximity sensing. Describes fabrication expense of rear-mounted electromagnetic digitizers. Electronic ink in windows 95.
PKCS11 API standard Version 1.0 for smartcards and other cryptographic tokens: single-user, mostly portable devices. Object-based in "C".
What it says: publics FTP servers on internet accessible by eMail.
Touchscreen sensor extends past edges of display for fixed buttons or keyboard. Touch sensor can be opaque on edges/margins. Compare with Wang Freestyle "The Guide"?
Handheld computer with slide-out/detachable keyboard: Mentions Momenta.
Bibliography only: contains reference to Ledeen Recognizer in Newman/Sproull textbook, other interactive editors (without character recognition): Application GUI using Ledeen recognizer.
Sales Automation application using Windows-based pen computer. Input into "letter guides", boxed character recognition input, makes distinction between character cells which are empty, and those which contain a space. Compare with Ward patent on simulated keyboard input using handwriting input.
Automatic software agents recognize text, but also figures, tablet, paragraphs of interest, via detectors and add information links (actions).
Touch panel displaying different pop-up menus depending on where it is touched: entry is selected by separate button when desired item is displayed.
Rotatable handwriting input widget/panel so that user can write/input from various directions, separate from orientation of touchscreen display. Compare with SIP panel?
Display can be rotated to laterally elongated position (landscape mode) or vertically elongated (portrait mode), control unit sets display format for windows. Saves orientation information when turned off. Compare with Radius Technology, Masimo 93?
Elastic matching (degree of one-dimensional deformation) with template for OCR optical character recognition. Mentions real-world preprocessing problems of difficult binarization, difficult separate from background. Samples taken from standard NIST OCR database.
Trapezoidal/rotational editing of part of a picture by defining four points of a trapezoid, and moving the four corners.
Optical mouse in pen/stylus form: light sensor in stylus, two colors of lights (or two filters), surface has grid of horizontal lines and vertical lines of different colors so that stylus/mouse can detect movement in X and Y by counting lines. States that can be used for writing (unlike mouse) because can detect movement in proximity, like many tablets.
Parsing of informal structure in electronic ink sketching in an electronic blackboard (whiteboard) system.
Case-study of designing touchscreen user interface for telephone, IBM project for pen-computing on a phone. Shows extensive use of paper mock-ups (sketches), simulation on Hypercard. Refers to notebook user-interface, rotate horizontal/vertical landscape/portrait orientation, visually similar to PenPoint (tabs, pen/stylus, etc.).
Touch sensors on keys of keyboard: user gestures (combinations of swipes) on space bar to enter/leave graphics/pointing mode for other mode. Mentions using two pointing keys for two hands: compare to Buxton bi-manual systems, multi-touch. Handicapped use by operating sensors with tongue. Grid array of sensors, scanned by rows and columns: optical touch sensor measures proximity of reflected light from small emitters: interpolation of signal intensity for higher resolution.
Elastic matching (see Tappert) applied to OCR of handwritten numerals, after thinning and line tracing.
Business tutorial on wireless, cellular, and other technologies: includes notes on EO/Hobbit pen computing, IBM Simon, PDAs with wireless communications.
Textbook, historical reference and tutorial on cryptography: includes C source code on MD5, IDEA, DES, etc. Listing of patents p. 447 for RSA and Diffie-Hellman key exchange.
email@example.com: long bibliography of pen/handwriting input and other GUI technologies.
Rufus language: Indexing of document (for retrieval) by parsing for implicit or semantic structure in the document: e.g. Jimmy Carter in Title, rather than Author or mentioned in passing.
Electrostatic/capacitive digitizer with cordless stylus or finger touch: mentions shielding between LCD and sensor (front-mounted), greater optical transmissivity due to optical properties of ITO shielding compared to touchscreens. Cordless stylus requires battery. References Windows 95 with Pen Extensions (Pen Windows). Joint product with NCR?
Electrostatic/capacitive digitizer with cordless stylus or finger touch. Describes capacitive sheet digitizer as "non-contacting resistive digitizer". Has good simplified description of resistive sheet tablets, ratiometric / normalizing coordinates.
Symbios manufacturing hardware controllers for Scriptel transparent tablet digitizers: Cites advantages of proximity sensing, no electromagnetic interference, cordless stylus over other PDA touchscreen technologies. Simulationeous pen and finger-touch detection: multi-touch.
Pointer device (touchscreen/stylus) movement in different directions to adjust contrast up/down and brightness up/down. Compare with marking menus? Initiated by key, or active while a button is held down.
Optical digitizer on "smart paper", with a (passive) pattern printed on the paper -- Compare to Anoto technology. Appears to use a something similar to a circular barcode / ringcode: provenance not given, might be interesting to find. Digitizer tablet "smart paper" that puts a printed code on paper, position is read by optical sensor in the stylus tip. Compare with technology by Anoto et al. Shows circular barcode optical target, doubled 1d Barcode target, square matrix target for "smart paper" Two styli pictured, no parallax correction (?) to optical image in optical digitizer.
Haptic/tactile feedback to a mouse, joystick, or other pointing device based on storing the window handle or other information in the input device and generating the feedback locally: faster than basing it on the information sent to the host.
Handwriting recognition on tablet with editing: editing symbols == editing gestures for text. Handwriting edit gestures of down-left, pigtail, etc. -- compare with PenPoint and Pen Computing gestures.
Visual programming: Semantic constraints may interfere during editing of diagrams / sketch editing / Visual languages, as editing may leave state inconsistent: proposes incremental validation of constraints.
Prograph graphical programming language.
Electronic organizer (PDA) with full keyboard, touchscreen. Fax modem. Touchscreen four-corner calibration. Touchscreen used as mouse (menus etc.), electronic ink. Some touchbuttons duplicate keyboard functionality e.g. "Store" button same as "Enter" key. Pop-up onscreen numeric keypad. VT-100 terminal emulation. Mentions emulation of PDA on PC display.
Sharp IQ-9200 PDA, dates approximate (1995)? has full keyboard in clamshell design, touchscreen with stylus for sketch input.
"spatial parser" to attempt to recognize implicit structures in how people lay out information: example is paper notecards, organized as horizontal and vertical lists, stacks, composite (list with additional "title card"), heaps. Compare with visual languages and 2D/two-dimensional parsing of mathematical expressions.
Encrypted container for protecting information rights (digital rights managements DRM). Specifications for DigiBox container (license server DRM) submitted to ANSI IISP Electronic Publishing Task Force EPUB, WG4. Models / language include purchase versus pay-per-use (metered). Clearinghouse for usage information and payments. Attached header includes transport key block, container table of contents. Encrypting 10% ten percent of video file makes it effectively uncopyable. Compare with ContentGuard, Stefik. Cites to superdistribution.
In a Wellner-type optical/augmented desk / touchscreen, a tag with a button and an LED flasher. The LED position can be determined in the camera image, the LED is modulated to send a recognizable ID, and the button can be used as user input. Used to tag physical objects so that they can have augmented virtual capabilities (e.g. tag on phone ties it to on-line dialer director).
First (?) version of OpenGL graphics library: includes subpixel drawing, alpha blending for transparency., Screen-door transparency == cheesy transparency.
Using pixel data to render transparent/overlay images in windows and objects, according to Z level of the windows or objects.
Touch sensor that makes 3D images of surfaces: rather than a thin sensor membrane, uses a tank or container filled with a liquid with a conformable membrane on top: pressed on an object, membrane with liquid conforms to shape of surface, video camera captures image of reflected light depending on angle of incidence with the (conformed) surface.
Abstract refers to transparent sensing surface over display, electronic ink, character recognition, and learning mode.
Abstract reads almost identically to Sklarew patent 4,972,496: this one refers to "terminal disclaimer".
Forms entry in handwriting recognition user interface, where a data-entry form field window pops up for entry, after entry of a pre-determined command using command/gesture symbols.
Abstract refers to transparent sensing surface over display, electronic ink, character recognition, and learning mode.
Abstract reads almost identically to Sklarew patent 4,972,496: this one refers to "terminal disclaimer".
Sklarew/Grid very specific to resistive digitizer, no proximity sense (?).
Tutorial on pre-multiplied and not pre-multiplied alpha blending, producing a shaped image with partial transparencies. Treatment of "over" operator (transparency) in Porter paper of 1984. Alpha is a fourth value "transparency" or "opacity" in addition to RGB values. Discussion of pixels modelled as points, or as square windows to which an object may partially contribute by crossing part of the square.
A pixel is a point sample, not a square or rectangle. Must represent contribution of all graphics that are behind it: may be overlapping circle with region of adjacent pixels, or of broader nearby pixels for reconstruction of image. Argument applies also to scanning and printing. Should not refer to shape of pixels (e.g. rectangular, non-square) but to non-square/non-uniform spacing of pixels.
Alpha blending: integral alpha, premultiplied alpha, digital printing, graphical compositing, matte / masking / matting blending. Traces history to analog-film compositing using a physical matte to combine images. History of blue-screen/green-screen matting, Petro Vlahos. Digital compositing an obvious simulation of known analog-film techniques, but digital matting (using alpha blending) not obvious.
General review of electrostatic/capacitive sensing, from simple proximity detection up to 3D and beyond. How to determine position using sensors in any particular configuration -- basically calculating or determining a correction table or formula to get the position of the object from the mix of sensor values, for whatever configuration you know the sensors are in. No references to commercial products e.g. Scriptel, academic references.
Self-4.0 user interface: object-oriented programming originated in attempt to make software model real-world behaviors, need to apply those principles more closely. reification of layout constraints (automatic layout) for user interface, composition through embedding. Software can be modified while running. Slots for actions on an object. Graphical objects referred to as morphs. Cites to Visual Basic, but not to PenApps by Slate.
Description of Newton application and OS architecture: slots, packages, interpreted bytecodes, no separate address spaces for different applications. Describes Newton as simplified version of "Self", frames take place of Self objects. Defines View system (Smalltalk 80), and MacApp. Newton applications delivered in packages, containing parts dispatched to the appropriate part of the OS when installed: e.g. application part, font use by application, special communications driver. Pages in virtual memory are compressed, and decompressed when loaded. Picture from Newton Toolkit: cites MacApp for views.
Discussion of NewtonScript for the Newton pen-computing PDA, based on prototype-based inheritance language SELF. Note: NewtonScript adapted from SELF's "parent slots". Self based on SMALLTALK.
NewtonScript prototype-based language for Newton development: describes historical antecedents. Visual Pen development environment for Newton Toolkit: view system and collection of slots. Browser weak at supporting non-UI code, really just a "frame hierarchy" editor. Inheritance (packages?) inspired by HyperCard. Also cites to Bay-Wei Change, 1993, Smalltalk-80, Self.
Tutorial on Visual Basic language and development environment.
WriteTouch electrostatic/capacitive digitizer from Scriptel supports either finger or stylus input: can do multi-touch? Announcement of cordless electrostatic digital pen using Symbios Logic mixed signal design with Scriptel.
References, Product information on Sony Magic Link PIC-1000 PDA "Personal Intelligent Communicator": hands-free cellphone option, touchscreen, but virtual keyboard only, no handwriting recognition. Magic Cap OS from General Magic.
Sony Magic Link PIC-1000 PDA "Personal Intelligent Communicator".
Study and statistical model (based on frequency digrams) of virtual keyboard typing speed on touchscreen with stylus. States more accurate and faster (overall) than handwriting recognition. QWERTY keyboard. Notes slipperiness of touchscreen, visual parallax. No citations to commercial pen computing systems (PenPoint, PenWindows, etc.).
Force/touch touchpad sensors on desktop/tabletop, may be used to create virtual input devices (virtual keyboard, menu, tablet, etc.), sense location of physical objects (phicons?). Identify/authenticate user by force-image/biometrics of palmprint, etc. (multi-touch / images of hand and fingers) Instrument/augment objects such as locks, floor.
Technical dictionary of computer terms 1994.
Sketch recognition for mechanical design: uses models of physical/mechanical constraints to produce actual designs from sketch. States modifies sketch until a design that functions in a specified manner.
Cited in Arvo00.
Electrostatic resistive sheet touchscreen digitizer, can sense both a stylus by a particular signal and a finger touch by capacitance at the same time: e.g. write with stylus while selecting menu with finger. Gives formula for shift and scale correction/calibration, but not quadrilateral calibration correction. multi-touch capable?
Semi-transparent "filter" viewer window can be dragged over other objects to show further/other information. Example: definitions of words in text, type of whitespace, zoomed view, additional information on a map. Figures show transparent window over text.
Semi-transparent "filter" viewer window has VOR "Viewing Operation Region", can be dragged over other objects to show further/other information. Example: definitions of words in text, type of whitespace, zoomed view, additional information on a map. Figures show transparent window over text.
MagicLens: visual transformations of part of a lower window when a second window is over it. Gives the effect of the second window being a lens: e.g. shade the color, magnify, show hidden lines/wireframe, show streets on a map, etc.
Cordless digitizer pen, sense stylus buttons by button changing resonance frequency of stylus.
Remote whiteboard touchscreen system combining LiveBoard and networked personal computers. Emphasizes utility of gestures when working with whiteboard conferencing, with alternative pop-up menus. Automatic recognition of gestures versus scribbles/drawing. Gestures include box/rectangle gesture to create document node, single connecting stroke to create links, stroke/flick in from outside of node, press-and-hold to start drag, pigtail/delete gesture. Refers to cordless pen/stylus of Xerox LiveBoard.
Open Windows / X Windows reference manual: includes vkbd(1) virtual keyboard/soft function keys display manager: display labels of function keys on screen with application-dependent labels automatically -- can use physical function keys or on-screen virtual keys input area.
Open Windows / X Windows reference manual.
Optical touchscreen, LED light emitters and photodetectors are arranged diagonally instead of X/Y. Light intensity reduced on corner emitters to reduce reflections.
Marking menus with stylus: animation feed back to user if mark (inadvertently) triggers a command gesture. Compare with press-and-hold, and stroke if do not wait.
Aviary system: Re-mapping of UI events (mouse button clicks) to other application events (or system events). Compare to press-and-hold event mapping in later Windows version.
Report on StarFire video project, showing speculative multi-touch user interfaces. Primary concerned with virtual video prototyping by means of "special effects" dramatized video production, which forces one to consider actual use cases in the course of building a shooting script for a film.
Essays on user-interface design, mentions 1994 "StarFire" video speculating on future touch-input portable computers, including a demonstration of a multi-touch (bimanual/two-hand) zoom gesture. Discussion on handwriting only mentions Graffiti by Palm. On touchscreens, states that poor accuracy (or visual parallax) causes users to slow down in an attempt to be more accurate: speculates on fiber-optic coupling plate (?) to reduce visual parallax.
Hardware user's manual for Toshiba T200 and T200CS pen computers -- mentions screen rotation, handwriting, etc. but does not name the operating system (PenPoint or Windows for Pen Computing).
Physical operational unit of Toshiba pen computer running Windows for Pen Computing 1.0 on top of MS/DOS 6.22. Photographs of screen rotation between landscape and portrait orientation, on-line gesture tutorials (with examples), Pen Palette pop-up handwriting bar for text entry, virtual keyboard in multiple configurations. SideKick 2.0. Base is Windows 3.1, with pen extensions.
Similar unit to Toshiba T200 80 Pen Windows 1.0, but color screen (better contrast on physical device and in photographs).
DataGlyphs: see other references. Compare to Anoto. Shows square matrix optical target, with asymmetric halftone dot patterns in cells of a digital halftone image.
Mentions papers and work direct manipulation / pen / stylus interfaces for mathematical input and editing (pp 27-30).
Accessibility access using multiple modes of input (equivalently) on touchscreen terminal: prototype application is access to technical documents database. Input by mouse, keyboard or touchscreen, presentation/output/display as graphics, large print/fonts, voice, inverted video / high contrast.
Marking menu keyboard: get a pop-up of a rotary/pid menu with letters/characters, arranged concentrically in multiple rings, or set of characters is magnified in the direction you move the stylus from the center. Hand occlusion / obscuring of menu dealt with by showing pop-up menu at an offset from pen tip: hand obscures nearly half of marking menu on a pen computer. Audio feedback (clicks) on the flick gestures at start and end. Visual feedback of selected character and idealized form of the flick gesture. Target can also be displayed continually, like a floating graphical keyboard.
Installation instructions for Apple T-Cube pie-menu (unistroke handwriting) text input.
ParcTab hand-held PDA with touchscreen and three physical buttons, designed to be held in the palm. Uses Unistroke character recognition, click/tap gestures on hierarchical menus. Tracks users by identifying location in office to a particular room via local infrared network. User can execute setup command to flip/rotate display by 180 degrees for left- or right-handed operation.
ParcTab hand-held PDA with touchscreen and three physical buttons, designed to be held in the palm. "Tab" refers to "small tablet computer" -- touchscreen computer. Uses Unistroke character recognition, click/tap gestures on hierarchical menus. Tracks users by identifying location in office to a particular room via local infrared network. User can execute setup command to flip/rotate display by 180 degrees for left- or right-handed operation.
Ultrasonic vibration of a few micrometers amplitude makes surface feel slick, less friction, due to small air film (hypothesized) under finger. Haptic feedback of stickiness, roughness, small protrusion for touchscreens, other surfaces.
Real-time electronic ink annotations while video is being made, uses as keys to video. Intent is to eliminate having to review the video later to log/annotate/index it. User takes notes, can circle a "keyboard" in the notes to use as index. Compare with Anoto smart-pen note taking (combined audio/electronic ink recording). Horizontal scratch-out gesture (not confused with M/W in handwriting recognition): keywords translated later manually by user typing?
Newton OS, much is on details of memory management Memory space is a single (large) linear address space but mapped in pages, protection among tasks: because not context mapping among multiple address spaces, thread/task switching is very fast, object IDs are just their linear address. (Novel at the time?) downloading of system updates, other software.
Mentions Freestyle, and asserts that people "like" paper compared to electronic documents. Cameras view paper desktop, read paper, also project images onto the desktop on top of the paper. Mostly a user-interface experiment in combining electronic simulation with real paper documents. Image subtraction and motion detection to locate hands etc. (Longish discussion of problem using a standard "C" image processing library from Modula, due to garbage collection).
LabVIEW Graphical programming language "G" for virtual instruments: has simulated mechanical action of boolean switches, both switching and latching (when pressed, when released, until released). Example of Global Switch.
About hand and body gestures of the form "put-that-there" gesticulation, rather than handwritten gestures as part of a computer GUI: refers in contrast to pen-based gestures with hand-held objects, instead of open-hand gestures (topic of thesis).
Bibliography of gesture recognition research: primarily about hand and body gestures of the form "put-that-there" gesticulation, rather than handwritten gestures as part of a computer GUI.
Panel discussion on gesture recognition research: primarily about hand and body gestures of the form "put-that-there", rather than handwritten gestures as part of a computer GUI. Edward Altman, Jelle Coutax, Sidney Fels, Wen Gao, Nobuo Hataoka, Bernard Hibbits, Volker Kuehn, Christoph Maggioni, Axel Mulder, Polly Pook, Alan Wexelblatt.
Gesture recognition for natural gestures (whole body gesticulation, empty-handed gestures/gesticulation), rather than pen-input gestures or handwriting recognition.
With mouse, users can point faster and more accurately if motion is vertical or horizontal than diagonal.
Multi-user information system, to use cordless pens, simultaneous tracking of multiple pens: 2..3 users enough in practice. Prototype uses resistive film with corded pen. Text handwriting recognition using CIC MacHandwriter. Wall mounting of prototype more awkward than original MacHandwriter hardware with desktop tablet. multi-touch with styli.
Simultaneous voice and electronic ink / handwriting recording in portable device: indexing of handwritten notes to audio recording. Compare with Wang Freestyle (cited) and to Anoto SmartPen/LiveScribe (later).
IBM Simon early "smartphone" with touchscreen. ROM-DOS operating system. GUI for specific applications "Navigator". Compare with EO/PenPoint with cellular phone? Refers to Simon form factor as a "brick".
Cirque Corporation GliePoint capacitive touchpad, developed 1991, commercial 1994. Acquired by Alps Electric 2003. Absolute position on touchpad converted to relative motion by subtraction of previous value.
Dylan: Competing language and IDE to NewtonScript for the Newton PDA: IDE released as technology release in 1995.
General Magic development of pen-based PDA/pocket digital communicator, using Telescript programming language to implement software agents. Compared to Apple Newton. Bill Atkinson, Andy Hertzfeld.
Resistive film digitizer, voltage gradient. Correction for edge / pincushion distortion by using a variety of connectors at the edges, with resistance corrections on the connectors. Transparent for touchscreen.
Secure network sockets: Secure Network Protocol in distributed security, predecessor to SSL key exchange and public-key encryption.
Scrollbar display highlight marks corresponding to where text had been found in a document. Compare with Russian patent to Gorbunov on the similar feature.
Xerox PARCTAB palm-sized mobile computer / PDA, worn like a pager. Three function buttons and stylus user interface on display touchscreen.
Secure processors: (co)processors with secure code in ROM, processor packed in epoxy with self-destruct lines, checksum on all loaded code. Compare with smartcards. See also Dyad system (secure DRM).
Integrated touchscreen/display device, connects wirelessly to Windows PC (Windows 95, Windows 3.1, PenWindows?). Image from http://billbuxton.com, 2012. Electronic file has additional references.
Recognize dwell/press-hold in stylus and distinguish between mouse / gesture / handwriting input mode: press-and-hold.
Silk Cursor using "silk stocking" effect: partial transparency to simulate (non-continuous) difference in depth instead of stereoscopic views. Touchscreen targeting accuracy?
Comparison study of 6DOF pointing devices. The "Bat": Polhemus 6DOF tracker with a handle; Spaceball isometric device; SpaceMaster; Cricket; SpaceMouse; MITS glove (space tracker with hand-compressible clutch). Remarks on limited generalization of conclusions in human factors research. For isometric devices (e.g. joystick), rate control better than position control. Investigation of control feel (feedback).
Handi: handsketch-based diagram editor in Express-G language, user-defined an pre-programmed unistroke gestures: one-stroke "T", circle, square, semicircle left/right gestures.
Human touch sensing on touchpad and other objects (furniture, location in room) using capacitance: field shunted to ground by human body, and field transmitted through conductive human body to stationary receivers. General remarks on capacitive sensing in general. Used to implement capacitive touchscreen ("finger-pointing mouse"). Can distinguish multiple persons (multi-touch). Signal-to-noise ratio increase with square root of samples averaged.
biblio93 (Prev page)
(Next page) biblio98
This compilation and all annotations are copyright © Jean Renard Ward, 1993, 2004, 2011, 2013, 2016.
I can be contacted at firstname.lastname@example.org.