Sorry your browser is not supported!

You are using an outdated browser that does not support modern web technologies, in order to use this site please update to a new browser.

Browsers supported include Chrome, FireFox, Safari, Opera, Internet Explorer 10+ or Microsoft Edge.

AppGameKit Classic Chat / [BUG] [2.0.15] set*ExtendedFontImage encoding

Author
Message
Scary Little Rabbit
14
Years of Service
User Offline
Joined: 4th Aug 2009
Location: Chelyabinsk, Russian Federation.
Posted: 16th Jan 2016 19:22
sorry for my previous report — it is not wrong at all, e.g. something like 'привет, мир' can be readed and writed smoothly. but it can't be displayed rightly because setTextDefaultExtendedFontImage() and setTextExtendedFontImage() are broken or something. looks like UTF8 is treated like CP1251 because I see 'привет, РјРёСЂ' instead 'привет, мир'.
error #1:
'too many stars, too many stares. disembody.'
WIP: MIND!! free fonts for your AGKs
Scary Little Rabbit
14
Years of Service
User Offline
Joined: 4th Aug 2009
Location: Chelyabinsk, Russian Federation.
Posted: 29th Feb 2016 16:57 Edited at: 29th Feb 2016 17:01
AGK 2.0.17 on Ubuntu 14.04



there's screenshot in attachment.
error #1:
'too many stars, too many stares. disembody.'
WIP: MIND!! free fonts for your AGKs

Attachments

Login to view attachments
Markus
Valued Member
20
Years of Service
User Offline
Joined: 10th Apr 2004
Location: Germany
Posted: 29th Feb 2016 18:12
u set the editor also to 1251?
AGK (Steam) V2.0.16 : Windows 10 Pro 64 Bit : AMD (15.30.1025) Radeon R7 265 : Mac mini OS X 10.10 (Yosemite)
Scraggle
Moderator
20
Years of Service
User Offline
Joined: 10th Jul 2003
Location: Yorkshire
Posted: 29th Feb 2016 18:14 Edited at: 29th Feb 2016 18:16
I replied to this in your previous post.
There is no support for UTF-8 in AGK. Crazy but true!

I am using Russian text (and many others) in my current project though, so there is a workaround - a horrible one but it works. The trouble is, it only works for displayed text and not input so although you can use the following workaround yourself for displayed fonts, you won't be able to ask for any user input.

What I have done is create a bitmap font for all Russian characters and then treated them as though they are English. Bare with me ....
Put all of your display text in a text file and then you will need to do a search/replace on every character so that the Russian alphabet is 'mapped' to the English alphabet.

For example:
In my project I have this text:
Quote: "HOW TO PLAY
COMPLETE A ROW TO UNLOCK IT.
UNLOCK ALL ROWS TO COMPLETE A LEVEL.
ALL GLYPHS IN A ROW OR COLUMN WILL BE REMOVED WHEN THAT ROW OR COLUMN IS COMPLETED."


Which translates in Russian to this:
Quote: "КАК ПЛАТИТЬ
ЧТОБЫ РАЗБЛОКИРОВАТЬ, ЗАПОЛНИ СТРОКУ
ЧТОБЫ ПРОЙТИ УРОВЕНЬ, РАЗБЛОКИРУЙ ВСЕ СТРОКИ.
СТРОКА И СТОЛБЕЦ БУДУТ УДАЛЕНЫ, КОГДА ВСЕ СИМВОЛЫ БУДУТ ЗАПОЛНЕНЫ."


But to get it to display correctly in AppGameKit, I convert the Russian text to this:
Quote: "KAK pdATNT?
yTO?| PAfbdOKNPOBAT}, fApOdHN CTPOKY
yTOb| pPOnTN YPOBEH}, PAfbdOKNPYn BCE CTPOKN.
CTPOKA N CTOdbEU bYDYT YDAdEH|, KOGDA BCE CNMBOd| bYDYT fApOdHEH|."


So now when AppGameKit tries to display a D it reads the bitmap font file and see that as a Д.

It is a horrible way to work but it gets the job done (for display text only) but it is, as far as I can tell, the only way to get Russian text to display in AppGameKit until TGC get around to supporting UTF-8.
AGK V2 user - Tier 1 (mostly)
Markus
Valued Member
20
Years of Service
User Offline
Joined: 10th Apr 2004
Location: Germany
Posted: 29th Feb 2016 18:52 Edited at: 29th Feb 2016 18:57
emm, print using a default bitmap font.
try this with a 1251 bitmap font
Quote: "
local fontimage as integer
fontimage = LoadImage ("Vivian48.png")
SetTextDefaultExtendedFontImage ( fontimage )
SetTextDefaultFontImage ( fontimage )

CreateText(1,"привет, мир!")

do
//print("привет, мир!") // hello world in ru

sync()
loop

"


https://en.wikipedia.org/wiki/Windows-1251
AGK (Steam) V2.0.16 : Windows 10 Pro 64 Bit : AMD (15.30.1025) Radeon R7 265 : Mac mini OS X 10.10 (Yosemite)

Attachments

Login to view attachments
Scary Little Rabbit
14
Years of Service
User Offline
Joined: 4th Aug 2009
Location: Chelyabinsk, Russian Federation.
Posted: 29th Feb 2016 21:02 Edited at: 29th Feb 2016 21:21
Markus wrote: "u set the editor also to 1251?"

nope. believe me you have no reason to set any other than UTF encoding on Linux. and as I remember I've got to convert all my projects back in the days when I've moved from Windows because of IDE/compiler glitches.
Markus wrote: "emm, print using a default bitmap font."

of course my example isn't quite good — but as you may see on screenshot there's 2 chars instead 1 all over there. if I set right font (already tested in the past since there was no this bug on AppGameKit 2.0.14 or something) then I see 'привет, РјРёСЂ!' as I said formerly.
Scraggle wrote: "so there is a workaround"

I don't want to implement workarounds for such a core thing. and don't forget there may be texts with mixed languages also.
error #1:
'too many stars, too many stares. disembody.'
WIP: MIND!! free fonts for your AGKs
Markus
Valued Member
20
Years of Service
User Offline
Joined: 10th Apr 2004
Location: Germany
Posted: 29th Feb 2016 21:44
Quote: " but as you may see on screenshot there's 2 chars instead 1 all over there."

ahh, yes, agree.
at work i have a small selfmade .net tool as converter for unicode to single byte in russian language. i made me a note, i can post later.
AGK (Steam) V2.0.17 : Windows 10 Pro 64 Bit : AMD (15.30.1025) Radeon R7 265 : Mac mini OS X 10.10 (Yosemite)
Paul Johnston
TGC Developer
21
Years of Service
User Offline
Joined: 16th Nov 2002
Location: United Kingdom
Posted: 1st Mar 2016 00:30
This is a known issue, we are hoping to add support for FreeType fonts in the future which will allow UTF-8 characters to be used.
Scary Little Rabbit
14
Years of Service
User Offline
Joined: 4th Aug 2009
Location: Chelyabinsk, Russian Federation.
Posted: 1st Mar 2016 01:20 Edited at: 1st Mar 2016 01:50
Paul Johnston wrote: "we are hoping to add support for FreeType fonts in the future which will allow UTF-8 characters to be used."

but why is it need to have FreeType fonts here? I believe there was some char codes casting/mapping over ASCII previously. now I also see asc() command returns -48 for "А", "Б", "В", etc but in some previous version of AppGameKit (sorry, I don't remember and don't have back up copies) the results was 192, 193, 194, etc for this sequence.
error #1:
'too many stars, too many stares. disembody.'
WIP: MIND!! free fonts for your AGKs
Paul Johnston
TGC Developer
21
Years of Service
User Offline
Joined: 16th Nov 2002
Location: United Kingdom
Posted: 1st Mar 2016 03:23
Quote: "now I also see asc() command returns -48 for "А", "Б", "В""

It looks like it returns signed chars instead of unsigned chars, if the value is negative you can add 256 to get the proper ASCII code, but I'll fix this in the next version so negative values are turned into unsigned values automatically.
Markus
Valued Member
20
Years of Service
User Offline
Joined: 10th Apr 2004
Location: Germany
Posted: 1st Mar 2016 07:44
@paul,
i think the problem here is if the editor is set to 1251 or 1252 (1 byte) it allows via copy/paste utf8 or unicode (1-4 bytes each char).
AGK (Steam) V2.0.17 : Windows 10 Pro 64 Bit : AMD (15.30.1025) Radeon R7 265 : Mac mini OS X 10.10 (Yosemite)
Markus
Valued Member
20
Years of Service
User Offline
Joined: 10th Apr 2004
Location: Germany
Posted: 1st Mar 2016 08:38 Edited at: 1st Mar 2016 08:40
here my tool (vs2010 source), it convert the clipboard to single byte and should match 1251.
AGK (Steam) V2.0.17 : Windows 10 Pro 64 Bit : AMD (15.30.1025) Radeon R7 265 : Mac mini OS X 10.10 (Yosemite)

Attachments

Login to view attachments
Scary Little Rabbit
14
Years of Service
User Offline
Joined: 4th Aug 2009
Location: Chelyabinsk, Russian Federation.
Posted: 1st Mar 2016 09:17
Paul Johnston wrote: "It looks like it returns signed chars instead of unsigned chars, if the value is negative you can add 256 to get the proper ASCII code"

note currently it returns -48 for each item in sequence of capitals of cyrillic alphabet. I haven't tested other chars.
Markus wrote: "here my tool"

thanks, man. I suppose there's similar converters availabale online too. but it would be hard to maintain source with such an abracdabra in it. as I said I prefer have the issue fixed.
error #1:
'too many stars, too many stares. disembody.'
WIP: MIND!! free fonts for your AGKs
Markus
Valued Member
20
Years of Service
User Offline
Joined: 10th Apr 2004
Location: Germany
Posted: 1st Mar 2016 11:33 Edited at: 1st Mar 2016 11:36
Quote: " as I said I prefer have the issue fixed."

agree.

as workaround
u just need a translate function, each char select case "и" asc=200 .. with the hope that Mid() works
means your own print command.
AGK (Steam) V2.0.17 : Windows 10 Pro 64 Bit : AMD (15.30.1025) Radeon R7 265 : Mac mini OS X 10.10 (Yosemite)
Scary Little Rabbit
14
Years of Service
User Offline
Joined: 4th Aug 2009
Location: Chelyabinsk, Russian Federation.
Posted: 1st Mar 2016 12:19
Markus wrote: "translate function"

yep, I've though about similar thing that's why I tried asc().
error #1:
'too many stars, too many stares. disembody.'
WIP: MIND!! free fonts for your AGKs
Markus
Valued Member
20
Years of Service
User Offline
Joined: 10th Apr 2004
Location: Germany
Posted: 1st Mar 2016 15:35
Quote: " that's why I tried"


ok, everything with multibyte in a string make problem with asc/mid/len/ ...

also my converted single bytes for windows 1251 have a wrong asc value (outside of 0-255)
AGK (Steam) V2.0.17 : Windows 10 Pro 64 Bit : AMD (15.30.1025) Radeon R7 265 : Mac mini OS X 10.10 (Yosemite)
Markus
Valued Member
20
Years of Service
User Offline
Joined: 10th Apr 2004
Location: Germany
Posted: 7th Mar 2016 15:25
.0.18
Changed the ASC command to return unsigned values instead of signed values, only affects ASCII codes above 127
AGK (Steam) V2.0.17 : Windows 10 Pro 64 Bit : AMD (15.30.1025) Radeon R7 265 : Mac mini OS X 10.10 (Yosemite)
Scary Little Rabbit
14
Years of Service
User Offline
Joined: 4th Aug 2009
Location: Chelyabinsk, Russian Federation.
Posted: 9th Mar 2016 09:40
AGK 2.0.18 on Ubuntu 14.04 64 bit
text output seems similar.
Scary Little Rabbit wrote: "note currently it returns -48 for each item in sequence of capitals of cyrillic alphabet."
now it's 208 for all chars which I've tested excepting non-capital 'ё'.

also I see that setDefaultFontImage() and setDefaultExtendedFontImage() don't work for print().

here's screenshot and project with custom Russo One font for testing.
error #1:
'too many stars, too many stares. disembody.'
WIP: MIND!! free fonts for your AGKs

Attachments

Login to view attachments
Paul Johnston
TGC Developer
21
Years of Service
User Offline
Joined: 16th Nov 2002
Location: United Kingdom
Posted: 9th Mar 2016 16:06
It looks like the characters are being input in UTF-8 format, which uses multiple bytes for some characters, of which the first character is 208 in this case. Using the menu option Document->Set Encoding->West European->Windows 1252 might fix it.
Markus
Valued Member
20
Years of Service
User Offline
Joined: 10th Apr 2004
Location: Germany
Posted: 9th Mar 2016 18:35 Edited at: 9th Mar 2016 18:36
@paul,
i believe he want a AscWide() command.
the old visual basic used always intern unicode for strings, but char encoding by default in ide was Windows-1252 = single byte codepage.
they provide a special command to get the char code for multybytes.
the agk ide allow copy/paste unicode into at string in a row, that make only sense if the agk compiler understand this too.

after copy/paste single bytes (Windows-1251 Chars) into agk ide and if me open the source file in notepad it seams its utf8 , if me save as ANSI its ok then in the agk ide.
to read a russian single byte string in the ide u need a font that support 1251 codepage. the Attachment screenshot shows 1252 chars, also the print command.
AGK (Steam) V2.0.17 : Windows 10 Pro 64 Bit : AMD (15.30.1025) Radeon R7 265 : Mac mini OS X 10.10 (Yosemite)

Attachments

Login to view attachments
Scary Little Rabbit
14
Years of Service
User Offline
Joined: 4th Aug 2009
Location: Chelyabinsk, Russian Federation.
Posted: 9th Mar 2016 19:06
Paul Johnston wrote: "Using the menu option Document->Set Encoding->West European->Windows 1252 might fix it."
actually it's 1251 for cyrillic, you'll get warning from IDE if you'll try to save document with cyrillic in 1252. and OK, it's working for 'english + cyrillic' documents, but what if I want to have there some Spanish words too — I can't save 'ñ' and other to 1251.

and encoding issue is relevant not for compile time only. imagine simle use case: you have a localization file with English, Russian and Spanish words and you load it at start up and you want be able to change language on the fly via clicking on a button. you may suggest to create few separate files for this case with specific encodings for all languages but I see it still isn't possible to get right code for some chars via asc() with that, e.g. it returns 241 for 'ñ' encoded in 1252 or ISO-8859-1 (right one is 164). maybe I need other encoding for Spanish? who knows? we're already in a kind of hell of (un)maintainability.

what would fix a problem for me finally it's a some new getUnicodeCodePoint() command to get literally Unicode code point of any char and some new rules for "font image subimages.txt" to point unicode code points to pieces of a font image.
error #1:
'too many stars, too many stares. disembody.'
WIP: MIND!! free fonts for your AGKs
Scary Little Rabbit
14
Years of Service
User Offline
Joined: 4th Aug 2009
Location: Chelyabinsk, Russian Federation.
Posted: 9th Mar 2016 19:14
here was no Markus' words when I started to write my answer.
Markus wrote: "i believe he want a AscWide() command."
yep, kind of. Unicode, Unicode is everywhere (it's not mine, just google's random one). UTF8 is real since 1992.
Markus wrote: "the agk ide allow copy/paste unicode into at string in a row, that make only sense if the agk compiler understand this too."
true.
error #1:
'too many stars, too many stares. disembody.'
WIP: MIND!! free fonts for your AGKs
Scraggle
Moderator
20
Years of Service
User Offline
Joined: 10th Jul 2003
Location: Yorkshire
Posted: 10th Mar 2016 07:41
Quote: "Using the menu option Document->Set Encoding->West European->Windows 1252 might fix it."


That's OK if your text is hard-coded but what if your text (like mine) is read in from a file?
We really need full unicode support for all text and files.
AGK V2 user - Tier 1 (mostly)
Markus
Valued Member
20
Years of Service
User Offline
Joined: 10th Apr 2004
Location: Germany
Posted: 10th Mar 2016 08:37
how about a new type. i think its cleaner than screw up the current string commands.
a as string
b as unicodestring

a as string1b
b as string4b
AGK (Steam) V2.0.17 : Windows 10 Pro 64 Bit : AMD (15.30.1025) Radeon R7 265 : Mac mini OS X 10.10 (Yosemite)
Richard Franklin
8
Years of Service
User Offline
Joined: 10th Mar 2016
Location:
Posted: 10th Mar 2016 10:29
i think the problem here is if the editor is set to 1251 or 1252 (1 byte) it allows via copy/paste utf8 or unicode (1-4 bytes each char)
Scary Little Rabbit
14
Years of Service
User Offline
Joined: 4th Aug 2009
Location: Chelyabinsk, Russian Federation.
Posted: 10th Mar 2016 11:16 Edited at: 10th Mar 2016 11:18
Markus wrote: "how about a new type"
it's overengineering. I suppose currently strings are stored in a kind of UCS-2 anyway. we able to read and write Unicode chars easily. what we need is to be able to a) get chars Unicode code points to parse strings and b) assign Unicode code points to parts of font image to output strings as AppGameKit text objects.
error #1:
'too many stars, too many stares. disembody.'
WIP: MIND!! free fonts for your AGKs
Scraggle
Moderator
20
Years of Service
User Offline
Joined: 10th Jul 2003
Location: Yorkshire
Posted: 10th Mar 2016 11:47
Quote: "what we need is to be able to a) get chars Unicode code points to parse strings and b) assign Unicode code points to parts of font image to output strings as AppGameKit text objects."


and c) be able to read the text from external Unicode files
AGK V2 user - Tier 1 (mostly)
Scary Little Rabbit
14
Years of Service
User Offline
Joined: 4th Aug 2009
Location: Chelyabinsk, Russian Federation.
Posted: 10th Mar 2016 13:19
Scraggle wrote: "c) be able to read the text from external Unicode files"
we able to do it already:
Scary Little Rabbit wrote: "we able to read and write Unicode chars easily."
I see no issues with that.
error #1:
'too many stars, too many stares. disembody.'
WIP: MIND!! free fonts for your AGKs
Scary Little Rabbit
14
Years of Service
User Offline
Joined: 4th Aug 2009
Location: Chelyabinsk, Russian Federation.
Posted: 14th Mar 2016 10:03
Scary Little Rabbit wrote: "some new rules for "font image subimages.txt" to point unicode code points to pieces of a font image."

e.g. "U+262E:0:0:12:36" for ☮.
error #1:
'too many stars, too many stares. disembody.'
WIP: MIND!! free fonts for your AGKs

Login to post a reply

Server time is: 2024-04-25 07:21:36
Your offset time is: 2024-04-25 07:21:36