Comparison of “copilot” models

At the moment GitHub Copilot is using ChatGPT-4 (based on info available). In general it works fine and boosts productivity a lot, but there are caveats that I want to highlight.

Let’s assume we have the following Spring Controller.

    @GetMapping("/api/preview/{id}")
    @ResponseBody
    public ResponseEntity<FileSystemResource> image(@PathVariable String id) {
        var idParts = id.split(":"); //must be "123:0"
        FileSystemResource fileResource = new FileSystemResource(
                previewDir.resolve(idParts[0] + "_" + idParts[1] + ".svg.gz"));
        return ResponseEntity
                .ok()
                .header(HttpHeaders.CONTENT_TYPE, "image/svg+xml")
                .header(HttpHeaders.CACHE_CONTROL, "max-age=600")
                .header(HttpHeaders.CONTENT_ENCODING, "gzip")
                .body(fileResource);
    }

Obviously I have to check format of id to ensure that it does not contain some security threats. I need to check that it fits format `digits”:”digits`

Let’s ask AI for help.

Github copilot

> check that id match pattern

public ResponseEntity<FileSystemResource> image(@PathVariable String id) {
    Pattern pattern = Pattern.compile("^\\d+:\\d+$");
    Matcher matcher = pattern.matcher(id);

    if (!matcher.matches()) {
        return ResponseEntity.badRequest().build();
    }

    var idParts = id.split(":"); //"123:0"
    FileSystemResource fileResource = new FileSystemResource(previewDir.resolve(
            Integer.parseInt(idParts[0]) + "." + Integer.parseInt(idParts[1]) +
                    ".svg.gz"));
    return ResponseEntity
            .ok()
            .header(HttpHeaders.CONTENT_TYPE, "image/svg+xml")
            .header(HttpHeaders.CACHE_CONTROL, "max-age=600")
            .header(HttpHeaders.CONTENT_ENCODING, "gzip")
            .body(fileResource);
}

Explicit check with pattern matcher. But, I know Spring has validators! Let’s try to convince copilot to change the code

> can it be done in declarative way with validator

Reply

Create a custom annotation:

import javax.validation.Constraint;
import javax.validation.Payload;
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

@Constraint(validatedBy = IdPatternValidator.class)
@Target({ ElementType.PARAMETER, ElementType.FIELD })
@Retention(RetentionPolicy.RUNTIME)
public @interface ValidIdPattern {
    String message() default "Invalid ID pattern";
    Class<?>[] groups() default {};
    Class<? extends Payload>[] payload() default {};
}

Create the validator:

import javax.validation.ConstraintValidator;
import javax.validation.ConstraintValidatorContext;
import java.util.regex.Pattern;

public class IdPatternValidator implements ConstraintValidator<ValidIdPattern, String> {
    private static final Pattern ID_PATTERN = Pattern.compile("^\\d+:\\d+$");

    @Override
    public void initialize(ValidIdPattern constraintAnnotation) {
    }

    @Override
    public boolean isValid(String value, ConstraintValidatorContext context) {
        return value != null && ID_PATTERN.matcher(value).matches();
    }
}

Update the controller to use the custom validator:

    @GetMapping("/api/preview/{id}")
    @ResponseBody
    public ResponseEntity<FileSystemResource> image(@PathVariable @Valid @ValidIdPattern String id) {
        var idParts = id.split(":"); //"123:0"
        FileSystemResource fileResource = new FileSystemResource(previewDir.resolve(
                Integer.parseInt(idParts[0]) + "." + Integer.parseInt(idParts[1]) +
                        ".svg.gz"));
        return ResponseEntity
                .ok()
                .header(HttpHeaders.CONTENT_TYPE, "image/svg+xml")
                .header(HttpHeaders.CACHE_CONTROL, "max-age=600")
                .header(HttpHeaders.CONTENT_ENCODING, "gzip")
                .body(fileResource);
    }

Well I expect something less wordy. I know there is a native way to check

> is there standard validator against regexp?

Ah yes!!!

    @GetMapping("/api/preview/{id}")
    @ResponseBody
    public ResponseEntity<FileSystemResource> image(@PathVariable @Pattern(regexp = "^\\d+:\\d+$") String id) {
        var idParts = id.split(":"); //"123:0"
        FileSystemResource fileResource = new FileSystemResource(previewDir.resolve(
                Integer.parseInt(idParts[0]) + "." + Integer.parseInt(idParts[1]) +
                        ".svg.gz"));
        return ResponseEntity
                .ok()
                .header(HttpHeaders.CONTENT_TYPE, "image/svg+xml")
                .header(HttpHeaders.CACHE_CONTROL, "max-age=600")
                .header(HttpHeaders.CONTENT_ENCODING, "gzip")
                .body(fileResource);
    }

Summary: OK I know spring and can direct Copilot. But for me it definitely faster to write it myself.

Chat GPT 4-o in browser window

In spring boot 3 application  check that controller id parameter fits pattern "digits:digits" - two numbers separated by ":"

    @GetMapping("/api/preview/{id}")
    @ResponseBody
    public ResponseEntity<FileSystemResource> image(@PathVariable String id) { 

 }

Reply (omitting explanation):

@GetMapping("/api/preview/{id:\\d+:\\d+}")
@ResponseBody
public ResponseEntity<FileSystemResource> image(@PathVariable String id) {
    // Your logic here
    return ResponseEntity.ok().body(/* Your FileSystemResource instance */);
}

Wow! this is exactly what I need. And is better/safer than parameter validation.

Conclusion(s): 4o (and especially o1-preview) writing significantly better code than gpt-4 used by copilot. At the same time lack of IDE integration makes it’s use difficult and most likely non practical.

Copilot(s) has huge potential for improvement. Looking forward to work with it once it switched to next gen model.

Do not rely on Copilot answer, use common sense. Answer could be outdated and/or do not follow best practices.