python django django-views django-forms django-templates

Why Django form shows UnicodeEncodeError in Persian language?

I have simple form as below:

forms.py:

from django import forms
class NameForm(forms.Form):
    your_name = forms.CharField(label='Your name', max_length=100)

views.py:

from django.http import HttpResponseRedirect
from django.shortcuts import render

from .forms import NameForm


def get_name(request):
    if request.method == 'POST':
        form = NameForm(request.POST)
        if form.is_valid():
            print(form.cleaned_data)
            return HttpResponseRedirect('/')
    else:
        form = NameForm()
    return render(request, 'form.html', {'form': form})

form.html

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Title</title>
</head>
<body>

<form action="/" method="post">
    {% csrf_token %}
    {{ form }}
    <input type="submit" value="Submit">
</form>
</body>
</html>

If I write English character and submit form, it works ok and execute print statement but if I write Persian character like'متن تست' it shows this error:

UnicodeEncodeError at /
'charmap' codec can't encode characters in position 132-136: character maps to <undefined>
Request Method: POST
Request URL:    http://127.0.0.1:8000/
Django Version: 4.1.7
Exception Type: UnicodeEncodeError
Exception Value:    
'charmap' codec can't encode characters in position 132-136: character maps to <undefined>
Exception Location: C:\Program Files\Python311\Lib\encodings\cp1252.py, line 19, in encode
Raised during:  myapp.views.get_name
Python Executable:  C:\python\test1\venv\Scripts\python.exe
Python Version: 3.11.2
Python Path:    
['C:\\python\\test1',
 'C:\\python\\test1',
 'C:\\Program Files\\JetBrains\\PyCharm '
 '2022.3.2\\plugins\\python\\helpers\\pycharm_display',
 'C:\\Program Files\\Python311\\python311.zip',
 'C:\\Program Files\\Python311\\DLLs',
 'C:\\Program Files\\Python311\\Lib',
 'C:\\Program Files\\Python311',
 'C:\\python\\test1\\venv',
 'C:\\python\\test1\\venv\\Lib\\site-packages',
 'C:\\Program Files\\JetBrains\\PyCharm '
 '2022.3.2\\plugins\\python\\helpers\\pycharm_matplotlib_backend']
Server time:    Tue, 14 Mar 2023 19:06:06 +0000

The string that could not be encoded/decoded was: lue="متن تست

What should I do?

I tried to use the below link but it didn't help me in Django:

UnicodeEncodeError: 'charmap' codec can't encode characters

Solution

The error occurs because the character encoding used by your system's console cannot handle the Persian characters. To fix this issue, you can set the default encoding to UTF-8 in your Django settings.py file as:

import sys
import io

sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding='utf-8')

This code sets the default encoding for standard output to UTF-8, which can handle non-ASCII characters.

Also, the HTML should have the following:

<meta charset="utf-8">

This tells the browser to use UTF-8 encoding for displaying the text.

Additionally, I'd also recommend you make a condition check when the form is not valid, say for example:

from django.shortcuts import redirect 

def get_name(request):
    if request.method == 'POST':
        form = NameForm(request.POST)
        if form.is_valid():
            print(form.cleaned_data)
            return HttpResponseRedirect('/')
        else:
            return redirect('some_error_page')
    else:
        form = NameForm()
    return render(request, 'form.html', {'form': form})