Together for Burundi!

by Kamusi Project USA

Project Report | Jul 19, 2016
A bunch of good news from Kamusi

By Martin Benjamin | Executive Director

Let's start with the bad news: the main Kamusi server is still not re-open to the public. Not being a systems administrator myself, and not having funds to pay a sysadmin, I'm taking a Zen approach to working with our valient volunteer as he sinks his own time and treasure into making it possible to actually share our work with the public again. When will we be fully back in business? It could be next week. It could be months. Zen.

Because, other than our server and funding woes, the rest of the past few months has been nothing but good news.

EmojiWorldBot (https://storebot.me/bot/emojiworldbot). We started this in April, and had the official release of version 1.0 for World Emoji Day, July 17. This is a dictionary between Emoji and more than 70 languages, and we have the tools to expand to many more. Why is this important? Well, Emoji are the common set of images that have been installed on, count 'em, 2 billion devices worldwide. This makes them a unique communications medium, and a shortcut for us to connect diverse languages, using Emoji as the pivot. We have a video to explain the basic features at https://youtu.be/eJV-MwCNeLs , and over 1100 happy users in the first few days of the release.
Chrome Extension. You want this. You really want this. Unfortunately, you can't have it yet because it is still on the internal server, but hey, Zen. When we can serve it, you will be able to double click any word on any page, and immediately look it up in its own language and the translation language of your choice. Here's the video: https://youtu.be/8Ve3aAC6YBY - there will soon be at least 30 languages interoperating. There are another 20-odd data sets that are open to us in principle, but we need signatures from various government ministries or senior professors, so... Zen.
Sign languages. I'm really happy about this: https://youtu.be/pzaKFVmreR8 . We're importing videos from the SignTyp project at the University of Connecticut for more than 20 different sign languages (and may have a source for another 25). The prototype works nicely. This will be a one-of-a-kind tool for communications between deaf people and any spoken language in the system. Want to help get it launched? Don't be Zen, send an email and volunteer to help with the DUCKS that we need to line up the data.
DUCKS stands for Data Unified Conceptual Knowledge Sets. This is our way of aligning data, so that we can find the equivalent terms across languages. We've got 100,000 definitions, and a game for matching to them - basically, asking "is this thing that is being called 'cup' something that you drink from, or something that you win at a tournament?" This lets us connect a ton of different languages and data sets, and get all our ducks in a row. However, it runs on our internal server, so access is by invitation only. You're invited - just send an email, and we'll get you hooked up.
WordUp! is our mobile app for getting terms from participants for languages far and wide, by giving them ducks-aligned concepts we don't yet have in their language. This needs a little polishing before we are ready to release it, but it is basically ready to go when the main server upgrades are finished.
We will also build a mobile app for dictionary lookups, combining the architecture of the Chrome extension and WordUp!. We need somewhere between a day and a week of coding power, and then you'll be able to do language-to-language lookups for any pair right from your phone, without using a browser. Of course, we need the server, and budget would be really helpful. Zen.
Pre:D. You probably don't want to read about "source side predisambiguation", so I'll just say that we've made some real progress on a program that will rock the socks off current approaches to getting the right vocabulary in machine translation. That is, where Google Translate has to guess in a sentence whether "She asked for a light for her cigarette" is referring to a flame or a lamp, Pre:D will know. Not sure when we'll have something you can use. New Years? Zen.
EatUp! We've made a lot of progress on an application for restaurant owners to input their menus in their local language, which translates the contents to the language spoken by the person who is tempted to eat there. Do not tell me that Google already does this. Don't go there, unless you think that "chicken tenders" are tasty people who take care of chickens, and I could go on all night. Again, they guess, and our point is to come at translation tasks on the basis of real knowledge, and do it for a whole lot of languages outside of the ones that have a pay-day. We're aiming to have this released by December.
EatUp! is also our platform for developing systems for other "controlled vocabulary" applications we want to implement as soon as possible, beginning with the FirstUp! app that has the terms that First Responders would need in an emergency, across languages, for situations like childbirth, poisoning, fires, heart attacks, etc. For the emergency vocabulary we've got access to some fantastic data sets that we can line up among a few dozen languages already, and farm to WordUp! for other languages. Guess where the hold up is. Budget. Server. Zen.

There's more in the pipeline, but I'll save it so there is something new to talk about in the next update.

What does this have to do with Burundi? A lot, really. WordUp!, in particular, is built with our African students in mind, because it pushes tasks to a system that works on the technology they are carrying around, without them incurring network costs. The game elements of EmojiWorldBot are similarly geared toward least common denominator technology. Many of the several dozen languages we are adding to the next version of the Bot are excluded languages from around Africa, and many of the data sets we are adding to DUCKS are from various African dictionary projects that have been mostly inaccessible before now. And the Kamusi Here! applications that build on the core logic of the Chrome extension will put all of this at the hands of students wherever they are, so they can type right into a chat (Facebook, WhatsApp, etc) something like, spanish:kirundi casa , and we will give them the lookup results, with any beginning or ending language that we have in the system. We haven't been able to move forward in the past few months with the students in Burundi because we haven't raised the full amount we need, and we haven't raised the full amount we need because the server has been offline and can't serve as a showcase, but the back end is ready for them. So we'll keep plugging away on the tools, praying away on the server, and, you know, Zen.

Happy Northern Summer/ Southern Chill,

Martin

Permalink

Apr 20, 2016
Kamusi Update, Spring 2016

By Martin Benjamin | Executive Director

Graph representation of multilingual Emoji data

We are almost ready to re-launch our public services, which have remained offline for the past several months. We had to make the difficult decision not to try to patch the old system, and instead focus our resources on building the new. The good news is that everything is falling into place on the development servers - the graph db model connects all of the elements of languages in highly complex but efficient ways that will open up new realms for what we can do with linguistic data. If we can protect the new system from the overwhelming robot traffic that consistently clogs the plumbing of the old, we are, dare I say it, close to being once again open for business. But I don't want to make any predictions regarding the timeframe.

In addition to our devoted long-term programmer plugging away in Johannesburg, we have several students working on various technical aspects of the project in Switzerland. I'm looking forward to flipping the switch soon for the mobile app that we will use to gather data across languages. We will also soon have a working prototype for an app to precisely translate restaurant menus, which will serve as the basis for other "controlled vocabulary" tools - the next one is planned to contain emergency terms for paramedics and other first responders to communicate with people in any language in situations from childbirth to poisoning. And, work continues on a tool we call "Pre:D" (for pre-disambiguation), that will lead to much more precise translations among many more languages than currently possible with services like Google Translate.

Likely first out of the gate, though, will be a dictionary for Emoji among more than 100 languages. We are using this constrained set of a few thousand concepts to test the new graph db, and to test a new way of gathering data from participants using Bots on the Telegram messaging platform. The Emoji data will be fully interlinked with the other data on Kamusi, so it will begin to come online for public access as soon as we come out of hibernation with the new server.

Regarding the work in Burundi, we are still struggling to raise the funds to get the next round of the project underway. The students' social networks are mostly within Burundi, where nobody has credit cards or extra funds to donate online, and I'm at wit's end about how to chase dollars and euros without a working website to demonstrate where we are heading. We'll plow through the Emoji data for Kirundi with the students at University of Ngozi in the next few weeks, and then try to figure out how to use that to showcase what we'll be able to do for the language when we have actual funds to support their work.

Please check the main site at kamusi.org from time to time to find out when we go live with the new system. If you see an error message like "page isn't working", our CPU overloaded again and stopped serving pages. If you see pictures of trucks, we're online, but limping with the old setup. However, if you see a bright and shiny announcement that we are back, then... thank you for your support through this tough period!

University of Ngozi, Burundi

Learning from native speakers one word at a time

Links:

Kamusi

Permalink

Jan 24, 2016
End of 2015 Update

By Martin Benjamin | Executive Director

Our plan vs. Q4 2015 reality

The last few months of 2015 were not a highlight in Kamusi history. We were not able to resolve the server problems that had been introduced by the massive growth in data that we'd brought in during the summer, basically because we still do not have the funds to implement the obvious (but expensive) technical fix needed to scale to hundreds or thousands of languages. This is incredibly frustrating, both because it is abysmal to keep giving people pages saying "server error" instead of giving them access to the shiny new dictionary model that was functioning before we fried the server, and because we've got a ton of new data and programming features piling up behind the server impasse.

When we break through the logjam, we have data ready to import that will bring us to about 10 million terms interlinked among 50 languages, totally unlike any other language resource. When we can afford management for the large volume of incoming data, we are also ready to introduce the games we've been developing for Facebook, and soon for mobile devices, that will be fun and compelling for speaker communities to grow data for their languages worldwide. The problem is that the "relational" database that makes chains from Language A to Language B to Language C, etc, was fine for relatively small amounts of data, but gets overwhelmed by millions of links. What we need to do is convert to a "graph" database, along the lines of what Google and Facebook use to handle their vast amounts of associations within their data. It's not a mystery how to do it, only how to pay for it.

On a positive note, we made good progress in building partnerships that will lead in great directions once we get beyond the current technical problem. I have to be coy because there are still some signatures that could take a while to gather - hopefully the agreements will fall into place before the next quarterly update.

I'm also happy to report that we've made significant progress with the Vietnamese data we've been working on. We've run into a bit of a problem that we ended up with close to 240,000 entries (190,000 English to Vietnamese and 50,000 monolingual Vietnamese) instead of the 10,000 that we'd anticipated for the first year, so processing the data has been a lot bigger task than we'd planned, especially with the server offline. However, this is a good kind of problem, and will work itself out once we are able to set the games in motion that will enable Vietnamese speakers to work the data into shape.

Finally, you will see that our focus on Global Giving at the moment is the campaign to raise funds for our Kirundi component. Kirundi is the language of Burundi, and for the past few years we've been working with the University of Ngozi in that country to produce a dictionary for their national language. It has been a fantastic relationship. We squeezed funds from the tail end of a grant, with which we were able to pay the university costs for about 10 students in Translation Studies each year. We trained the students in how to write dictionary entries and in the necessary computer skills, and they produced the beginnings of a great free dictionary for their country. However, two bad things have happened this year. First, our funding pot for the project ran completely dry. Second, Burundi has been in political turmoil, so the university has not been able to operate as normal. Conditions in Ngozi seem to be stabilizing, so the students are ready to get back to work. However, because we've scraped the bottom of the barrel with our bank account, we cannot bring on the next round of students until we have funds on hand to support their work. It's remarkably low in dollar/euro/sterling terms to fund a student for a year to both pay for their studies and pay for their contributions to Kamusi - just $300 per student - so we are hoping that this campaign can cover their stipends and the other costs associated with making the project run in Burundi. Please watch their video on the Global Giving website, and let's pull together to see what they can do for the students of their country!

Thanks for your continued support, and best wishes for 2016,

Martin

Permalink

‹ 1 2 3 4 5 ›

About Project Reports

Project reports on GlobalGiving are posted directly to globalgiving.org by Project Leaders as they are completed, generally every 3-4 months. To protect the integrity of these documents, GlobalGiving does not alter them; therefore you may find some language or formatting issues.

If you donate to this project or have donated to this project, you can receive an email when this project posts a report. You can also subscribe for reports without donating.